Max level shown:
1 [ Nov 28 01:33:03 Enabled. ]
2 [ Nov 28 01:33:03 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Nov 28 01:33:03 Method "start" exited with status 0. ]
4 [ Nov 28 01:33:03 Rereading configuration. ]
5 [ Nov 28 01:33:03 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72024-11-28T01:33:03.744ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:113
82024-11-28T01:33:03.745ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:255
92024-11-28T01:33:03.798ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:262
102024-11-28T01:33:03.799ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
112024-11-28T01:33:03.914ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
122024-11-28T01:33:03.922ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
132024-11-28T01:33:03.930ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:272 path = /var/run/oxide/
142024-11-28T01:33:04.534ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fdb0:18c0:4d0d:9fb2::, width: 64 }
152024-11-28T01:33:04.677ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:126
162024-11-28T01:33:04.677ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:138
172024-11-28T01:33:04.677ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:151
182024-11-28T01:33:04.677ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:173 nongimlet_observed_disks = [] sled_mode = Scrimlet { asic: SoftNpuZone }
192024-11-28T01:33:04.677ZINFOSledAgent (DumpSetup-worker): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:520
202024-11-28T01:33:04.677ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:717
212024-11-28T01:33:04.680ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "buskin", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:624
222024-11-28T01:33:04.680ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:766
232024-11-28T01:33:04.680ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:187
242024-11-28T01:33:04.680ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:825
252024-11-28T01:33:04.680ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_0.vdev
262024-11-28T01:33:04.680ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = None
272024-11-28T01:33:04.680ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
282024-11-28T01:33:04.680ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
292024-11-28T01:33:04.680ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } path = /scratch/m2_0.vdev
302024-11-28T01:33:04.693ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_0.vdev file = sled-hardware/src/disk.rs:367
312024-11-28T01:33:04.693ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = 2d6a91dc-89a8-488e-9efc-6f5bec9b10d5 (zpool)
322024-11-28T01:33:04.818ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 2d6a91dc-89a8-488e-9efc-6f5bec9b10d5 (zpool), kind: Internal }
332024-11-28T01:33:05.399ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:603
342024-11-28T01:33:05.399ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_1.vdev
352024-11-28T01:33:05.399ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
362024-11-28T01:33:05.399ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
372024-11-28T01:33:05.399ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
382024-11-28T01:33:05.399ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
392024-11-28T01:33:05.399ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } path = /scratch/m2_1.vdev
402024-11-28T01:33:05.408ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_1.vdev file = sled-hardware/src/disk.rs:367
412024-11-28T01:33:05.408ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = 7f63c436-f542-43a4-9af6-de72a4c8391f (zpool)
422024-11-28T01:33:05.522ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 7f63c436-f542-43a4-9af6-de72a4c8391f (zpool), kind: Internal }
432024-11-28T01:33:06.100ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_0.vdev
442024-11-28T01:33:06.100ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 2d6a91dc-89a8-488e-9efc-6f5bec9b10d5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 7f63c436-f542-43a4-9af6-de72a4c8391f (zpool), kind: Internal } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
452024-11-28T01:33:06.100ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
462024-11-28T01:33:06.100ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
472024-11-28T01:33:06.100ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
482024-11-28T01:33:06.100ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
492024-11-28T01:33:06.100ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_1.vdev
502024-11-28T01:33:06.100ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
512024-11-28T01:33:06.100ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
522024-11-28T01:33:06.100ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 2d6a91dc-89a8-488e-9efc-6f5bec9b10d5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 7f63c436-f542-43a4-9af6-de72a4c8391f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
532024-11-28T01:33:06.100ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
542024-11-28T01:33:06.100ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
552024-11-28T01:33:06.100ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_2.vdev
562024-11-28T01:33:06.100ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 2d6a91dc-89a8-488e-9efc-6f5bec9b10d5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 7f63c436-f542-43a4-9af6-de72a4c8391f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
572024-11-28T01:33:06.100ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
582024-11-28T01:33:06.100ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
592024-11-28T01:33:06.100ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
602024-11-28T01:33:06.100ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
612024-11-28T01:33:06.100ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_3.vdev
622024-11-28T01:33:06.100ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 2d6a91dc-89a8-488e-9efc-6f5bec9b10d5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 7f63c436-f542-43a4-9af6-de72a4c8391f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
632024-11-28T01:33:06.100ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
642024-11-28T01:33:06.100ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
652024-11-28T01:33:06.100ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
662024-11-28T01:33:06.100ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
672024-11-28T01:33:06.100ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_4.vdev
682024-11-28T01:33:06.100ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
692024-11-28T01:33:06.100ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
702024-11-28T01:33:06.100ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 2d6a91dc-89a8-488e-9efc-6f5bec9b10d5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 7f63c436-f542-43a4-9af6-de72a4c8391f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
712024-11-28T01:33:06.100ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
722024-11-28T01:33:06.100ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
732024-11-28T01:33:06.100ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_5.vdev
742024-11-28T01:33:06.100ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 2d6a91dc-89a8-488e-9efc-6f5bec9b10d5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 7f63c436-f542-43a4-9af6-de72a4c8391f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
752024-11-28T01:33:06.101ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
762024-11-28T01:33:06.101ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
772024-11-28T01:33:06.101ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
782024-11-28T01:33:06.101ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
792024-11-28T01:33:06.101ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_6.vdev
802024-11-28T01:33:06.101ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 2d6a91dc-89a8-488e-9efc-6f5bec9b10d5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 7f63c436-f542-43a4-9af6-de72a4c8391f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
812024-11-28T01:33:06.101ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
822024-11-28T01:33:06.101ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
832024-11-28T01:33:06.101ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
842024-11-28T01:33:06.101ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
852024-11-28T01:33:06.101ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_7.vdev
862024-11-28T01:33:06.101ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
872024-11-28T01:33:06.101ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 2d6a91dc-89a8-488e-9efc-6f5bec9b10d5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 7f63c436-f542-43a4-9af6-de72a4c8391f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
882024-11-28T01:33:06.101ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
892024-11-28T01:33:06.101ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
902024-11-28T01:33:06.101ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
912024-11-28T01:33:06.101ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_8.vdev
922024-11-28T01:33:06.101ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
932024-11-28T01:33:06.101ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
942024-11-28T01:33:06.101ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:97
952024-11-28T01:33:06.101ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 2d6a91dc-89a8-488e-9efc-6f5bec9b10d5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 7f63c436-f542-43a4-9af6-de72a4c8391f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
962024-11-28T01:33:06.101ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-agent/src/long_running_tasks.rs:99
972024-11-28T01:33:06.101ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
982024-11-28T01:33:06.101ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
992024-11-28T01:33:06.101ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1002024-11-28T01:33:06.101ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 2d6a91dc-89a8-488e-9efc-6f5bec9b10d5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 7f63c436-f542-43a4-9af6-de72a4c8391f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
1012024-11-28T01:33:06.101ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
1022024-11-28T01:33:06.101ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:211
1032024-11-28T01:33:06.101ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1042024-11-28T01:33:06.101ZINFOSledAgent (bootstore): No ledger in /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1052024-11-28T01:33:06.101ZINFOSledAgent (bootstore): No ledger in /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1062024-11-28T01:33:06.101ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-buskin-i86pc
1072024-11-28T01:33:06.101ZINFOSledAgent (bootstore): No ledger in /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1082024-11-28T01:33:06.101ZINFOSledAgent (bootstore): No ledger in /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1092024-11-28T01:33:06.101ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-buskin-i86pc
1102024-11-28T01:33:06.101ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:216
1112024-11-28T01:33:06.101ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:231
1122024-11-28T01:33:06.112ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:736
1132024-11-28T01:33:06.112ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 2411, tv_nsec: 329439072 } time_to_next_cleanup = 599.999945709s
1142024-11-28T01:33:06.112ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3947
1152024-11-28T01:33:06.112ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1162024-11-28T01:33:06.112ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/sled-agent-request.json file = common/src/ledger.rs:188
1172024-11-28T01:33:06.112ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/sled-agent-request.json file = common/src/ledger.rs:188
1182024-11-28T01:33:06.115ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80
1192024-11-28T01:33:06.117ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:51 local_addr = [fdb0:18c0:4d0d:9fb2::1]:12346
1202024-11-28T01:33:06.118ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.1.164, last: 10.151.1.173, }, ), ], external_dns_ips: [ 10.151.1.164, 10.151.1.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.1.100, vlan_id: None, rib_priority: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.1.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: Some( None, ), bgp_peers: [], autoneg: false, lldp: None, tx_eq: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, } file = sled-agent/src/rack_setup/service.rs:1187
1212024-11-28T01:33:06.118ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1222024-11-28T01:33:06.118ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1232024-11-28T01:33:06.118ZINFOSledAgent (RSS): No ledger in /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1242024-11-28T01:33:06.118ZINFOSledAgent (RSS): No ledger in /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1252024-11-28T01:33:06.118ZINFOSledAgent (RSS): RSS configuration has not been fully applied yet file = sled-agent/src/rack_setup/service.rs:1250
1262024-11-28T01:33:06.118ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1272024-11-28T01:33:06.118ZINFOSledAgent (RSS): No ledger in /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/rss-sled-plan.json file = common/src/ledger.rs:188
1282024-11-28T01:33:06.118ZINFOSledAgent (RSS): No ledger in /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/rss-sled-plan.json file = common/src/ledger.rs:188
1292024-11-28T01:33:06.118ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:1292
1302024-11-28T01:33:06.118ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:156
1312024-11-28T01:33:06.118ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:9fb2::1 file = sled-agent/src/rack_setup/plan/sled.rs:129
1322024-11-28T01:33:06.118ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1332024-11-28T01:33:06.118ZINFOSledAgent (RSS): Writing ledger to /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1342024-11-28T01:33:06.118ZINFOSledAgent (RSS): Writing ledger to /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1352024-11-28T01:33:06.118ZINFOSledAgent (RSS): Sled plan written to storage: Plan { rack_id: 847c32d1-2da6-4de4-82bb-09f10249a7c3, sleds: { [fdb0:18c0:4d0d:9fb2::1]:12346: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 12368a57-a643-445a-b32d-da10700bd20e (sled), rack_id: 847c32d1-2da6-4de4-82bb-09f10249a7c3, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64, }, }, }, }, }, config: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.1.164, last: 10.151.1.173, }, ), ], external_dns_ips: [ 10.151.1.164, 10.151.1.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.1.100, vlan_id: None, rib_priority: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.1.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: Some( None, ), bgp_peers: [], autoneg: false, lldp: None, tx_eq: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, }, } file = sled-agent/src/rack_setup/plan/sled.rs:176
1362024-11-28T01:33:06.118ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:1326
1372024-11-28T01:33:06.118ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-buskin-i86pc
1382024-11-28T01:33:06.118ZINFOSledAgent (bootstore): Writing ledger to /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
1392024-11-28T01:33:06.119ZINFOSledAgent (bootstore): Writing ledger to /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
1402024-11-28T01:33:06.119ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-buskin-i86pc
1412024-11-28T01:33:06.119ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:232 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 12368a57-a643-445a-b32d-da10700bd20e (sled), rack_id: 847c32d1-2da6-4de4-82bb-09f10249a7c3, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } target_sled = [fdb0:18c0:4d0d:9fb2::1]:12346
1422024-11-28T01:33:06.119ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:4244
1432024-11-28T01:33:06.119ZINFOSledAgent (ServiceManager): Starting switch zone file = sled-agent/src/services.rs:4644
1442024-11-28T01:33:06.124ZINFOSledAgent (SledAgentSprocketsClient): Loaded keys and certs BootstrapAgentClient = [fdb0:18c0:4d0d:9fb2::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/client.rs:50
1452024-11-28T01:33:06.131ZINFOSledAgent (SledAgentSprocketsClient): Certificate chain verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:9fb2::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1462024-11-28T01:33:06.131ZINFOSledAgent (SledAgentSprocketsClient): Signature verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:9fb2::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1472024-11-28T01:33:06.135ZINFOSledAgent (SledAgentSprocketsServer): Certificate chain verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1482024-11-28T01:33:06.135ZINFOSledAgent (SledAgentSprocketsServer): Signature verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1492024-11-28T01:33:06.135ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:74 remote_addr = [fdb0:18c0:4d0d:9fb2::1]:60655
1502024-11-28T01:33:06.135ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 12368a57-a643-445a-b32d-da10700bd20e (sled), rack_id: 847c32d1-2da6-4de4-82bb-09f10249a7c3, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } file = sled-agent/src/bootstrap/server.rs:362
1512024-11-28T01:33:06.135ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:377
1522024-11-28T01:33:06.135ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:44
1532024-11-28T01:33:06.135ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1542024-11-28T01:33:06.135ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:603
1552024-11-28T01:33:06.135ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:101::, width: 64 }
1562024-11-28T01:33:06.135ZINFOSledAgent (StorageManager): Loading storage config file = sled-storage/src/manager.rs:809
1572024-11-28T01:33:06.135ZINFOSledAgent (StorageManager): No ledger in /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1582024-11-28T01:33:06.135ZINFOSledAgent (StorageManager): No ledger in /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1592024-11-28T01:33:06.135ZINFOSledAgent (StorageManager): No ledger of physical disks exists file = sled-storage/src/manager.rs:798
1602024-11-28T01:33:06.135ZINFOSledAgent (StorageManager): KeyManager ready, but no ledger detected file = sled-storage/src/manager.rs:826
1612024-11-28T01:33:06.135ZINFOSledAgent (StorageManager): automatically managing disks file = sled-storage/src/manager.rs:592
1622024-11-28T01:33:06.136ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
1632024-11-28T01:33:06.136ZINFOSledAgent (StorageManager): Successfully synchronized disks without error file = sled-storage/src/manager.rs:697 result = DisksManagementResult { status: [] }
1642024-11-28T01:33:06.152ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1652024-11-28T01:33:06.174ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:325
1662024-11-28T01:33:06.191ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:363
1672024-11-28T01:33:06.219ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:415 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
1682024-11-28T01:33:06.219ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1692024-11-28T01:33:06.219ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:427 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
1702024-11-28T01:33:06.219ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1712024-11-28T01:33:06.222ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 85.099559ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "4527b563-8297-4352-97f5-92eb0aa6cc7b", "content-length": "124", "date": "Thu, 28 Nov 2024 01:33:06 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "4527b563-8297-4352-97f5-92eb0aa6cc7b" } file = clients/ddm-admin-client/src/lib.rs:177
1722024-11-28T01:33:06.232ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1732024-11-28T01:33:06.263ZINFOSledAgent: successfully created encrypted zvol: name="oxi_2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1742024-11-28T01:33:06.263ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/swap" file = sled-agent/src/swap_device.rs:96
1752024-11-28T01:33:06.268ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:442 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
1762024-11-28T01:33:06.268ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:128
1772024-11-28T01:33:06.268ZINFOSledAgent (BackingFs): Ensuring dataset oxi_2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/backing/fmd file = sled-agent/src/backing_fs.rs:138
1782024-11-28T01:33:06.308ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1792024-11-28T01:33:06.309ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 102.574247ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "328fdcbd-7f00-4da1-8f3f-58907979de54", "content-length": "124", "date": "Thu, 28 Nov 2024 01:33:06 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "328fdcbd-7f00-4da1-8f3f-58907979de54" } file = clients/ddm-admin-client/src/lib.rs:177
1802024-11-28T01:33:06.366ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:173
1812024-11-28T01:33:06.412ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1822024-11-28T01:33:06.413ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 108.364461ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "184e4220-6b47-4503-ac92-afe67103480d", "content-length": "124", "date": "Thu, 28 Nov 2024 01:33:06 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "184e4220-6b47-4503-ac92-afe67103480d" } file = clients/ddm-admin-client/src/lib.rs:177
1832024-11-28T01:33:06.522ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1842024-11-28T01:33:06.523ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 244.712063ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "f5f257ab-8116-4dee-a828-cff0e31e5eb1", "content-length": "124", "date": "Thu, 28 Nov 2024 01:33:06 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "f5f257ab-8116-4dee-a828-cff0e31e5eb1" } file = clients/ddm-admin-client/src/lib.rs:177
1852024-11-28T01:33:06.584ZINFOSledAgent (BackingFs): Mounting oxi_2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:181
1862024-11-28T01:33:06.597ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:189
1872024-11-28T01:33:06.597ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:189
1882024-11-28T01:33:06.597ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:189
1892024-11-28T01:33:06.597ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:195
1902024-11-28T01:33:06.768ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1912024-11-28T01:33:06.769ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 194.762808ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "ca8fdae7-4426-4b6e-a5df-b0b2c0b9248f", "content-length": "124", "date": "Thu, 28 Nov 2024 01:33:06 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "ca8fdae7-4426-4b6e-a5df-b0b2c0b9248f" } file = clients/ddm-admin-client/src/lib.rs:177
1922024-11-28T01:33:06.964ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1932024-11-28T01:33:06.965ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 135.427336ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "812ed640-0593-45b8-94e1-26e6196be5db", "content-length": "124", "date": "Thu, 28 Nov 2024 01:33:06 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "812ed640-0593-45b8-94e1-26e6196be5db" } file = clients/ddm-admin-client/src/lib.rs:177
1942024-11-28T01:33:07.101ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1952024-11-28T01:33:07.102ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 315.98929ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "ca21bc4c-814f-4977-a37a-3d3393bd9663", "content-length": "124", "date": "Thu, 28 Nov 2024 01:33:06 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "ca21bc4c-814f-4977-a37a-3d3393bd9663" } file = clients/ddm-admin-client/src/lib.rs:177
1962024-11-28T01:33:07.358ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:93 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
1972024-11-28T01:33:07.363ZINFOSledAgent (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:45191 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
1982024-11-28T01:33:07.363ZINFOSledAgent (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
1992024-11-28T01:33:07.363ZINFOSledAgent (producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::1]:45191 file = oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 12368a57-a643-445a-b32d-da10700bd20e sled_id = 12368a57-a643-445a-b32d-da10700bd20e
2002024-11-28T01:33:07.364ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 287.420472ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
2012024-11-28T01:33:07.365ZINFOSledAgent (VmmReservoirManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/vmm_reservoir.rs:267 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
2022024-11-28T01:33:07.365ZINFOSledAgent (VmmReservoirManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:275 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
2032024-11-28T01:33:07.365ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:920 underlay_address = fd00:1122:3344:101::1
2042024-11-28T01:33:07.365ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
2052024-11-28T01:33:07.418ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
2062024-11-28T01:33:07.653ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 460.949805ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
2072024-11-28T01:33:08.116ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 1.02806044s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
2082024-11-28T01:33:09.146ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 2.743821072s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
2092024-11-28T01:33:09.852ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2668
2102024-11-28T01:33:09.852ZINFOSledAgent (ServiceManager): Setting up lldpd service file = sled-agent/src/services.rs:2913
2112024-11-28T01:33:09.852ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2538
2122024-11-28T01:33:09.852ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2585
2132024-11-28T01:33:09.852ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:2994
2142024-11-28T01:33:09.852ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:3039
2152024-11-28T01:33:09.854ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2838
2162024-11-28T01:33:09.856ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2579
2172024-11-28T01:33:09.856ZINFOSledAgent (ServiceManager): Profile for oxz_switch: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/mgs"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12225'/> <propval type="astring" name="id" value='2c3cbc16-f8cc-4101-9e57-039fde27fada'/> <propval type="astring" name="rack_id" value='847c32d1-2da6-4de4-82bb-09f10249a7c3'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/wicketd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12226'/> <propval type="astring" name="artifact-address" value='[fdb0:18c0:4d0d:9fb2::2]:12227'/> <propval type="astring" name="baseboard-file" value='/opt/oxide/baseboard.json'/> <propval type="astring" name="mgs-address" value='[::1]:12225'/> <propval type="astring" name="nexus-proxy-address" value='[::]:12229'/> <propval type="astring" name="rack-subnet" value='fd00:1122:3344::'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/switch_zone_setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="gz_local_link_addr" value='fe80::8:20ff:fe44:f56a'/> <property type="astring" name="link_local_links"> <astring_list> <value_node value="tfportrear0_0" /> <value_node value="oxBootstrap0" /> </astring_list> </property> <propval type="astring" name="bootstrap_addr" value='fdb0:18c0:4d0d:9fb2::2'/> <propval type="astring" name="bootstrap_vnic" value='oxBootstrap0'/> <propval type="astring" name="baseboard_info" value='{"type":"pc","identifier":"buskin","model":"i86pc"}'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/dendrite"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="rack_id" value='847c32d1-2da6-4de4-82bb-09f10249a7c3'/> <propval type="astring" name="sled_id" value='12368a57-a643-445a-b32d-da10700bd20e'/> <propval type="astring" name="sled_model" value='i86pc'/> <propval type="astring" name="sled_serial" value='buskin'/> <propval type="astring" name="sled_revision" value='0'/> <propval type="astring" name="address" value='[::1]:12224'/> <propval type="astring" name="front_ports" value='1'/> <propval type="astring" name="rear_ports" value='1'/> <propval type="astring" name="port_config" value='/opt/oxide/dendrite/misc/softnpu_single_sled_config.toml'/> <propval type="astring" name="mgmt" value='uds'/> <propval type="astring" name="uds_path" value='/opt/softnpu/stuff'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/tfport"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="host" value='[::1]'/> <propval type="astring" name="port" value='12224'/> <propval type="astring" name="flags" value='--sync-only'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/lldpd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="board_rev" value='softnpu_front_1_rear_1'/> <propval type="astring" name="scrimlet_id" value='buskin'/> <propval type="astring" name="scrimlet_model" value='i86pc'/> <propval type="astring" name="address" value='[::1]:12230'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/pumpkind"> </service> <service version="1" type="service" name="oxide/mgd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_uuid" value='12368a57-a643-445a-b32d-da10700bd20e'/> <propval type="astring" name="rack_uuid" value='847c32d1-2da6-4de4-82bb-09f10249a7c3'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/mg-ddm"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="mode" value='transit'/> <propval type="astring" name="dendrite" value='true'/> <propval type="astring" name="sled_uuid" value='12368a57-a643-445a-b32d-da10700bd20e'/> <propval type="astring" name="rack_uuid" value='847c32d1-2da6-4de4-82bb-09f10249a7c3'/> <propval type="astring" name="interfaces" value='tfportrear0_0/ll'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/uplink"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
2182024-11-28T01:33:09.856ZINFOSledAgent (ServiceManager): Booting oxz_switch zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_switch
2192024-11-28T01:33:10.185ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 43.667965ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2202024-11-28T01:33:10.402ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 95.770997ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2212024-11-28T01:33:10.576ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 111.014025ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2222024-11-28T01:33:10.743ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 315.793786ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2232024-11-28T01:33:11.086ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.145940235s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2242024-11-28T01:33:11.892ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 5.311584404s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
2252024-11-28T01:33:12.255ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 546.303025ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2262024-11-28T01:33:12.862ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.455881163s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2272024-11-28T01:33:14.533ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 686.433603ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2282024-11-28T01:33:15.233ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 978.859117ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2292024-11-28T01:33:16.515ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/12368a57-a643-445a-b32d-da10700bd20e) file = sled-agent/src/nexus.rs:527 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
2302024-11-28T01:33:16.545ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2312024-11-28T01:33:16.545ZINFOSledAgent (dropshot (Repo Depot)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:12348 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
2322024-11-28T01:33:16.546ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:12345
2332024-11-28T01:33:16.546ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:432
2342024-11-28T01:33:16.546ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2352024-11-28T01:33:16.546ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2362024-11-28T01:33:16.547ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2372024-11-28T01:33:16.547ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:138
2382024-11-28T01:33:16.547ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56 }, infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [PortConfigV2 { routes: [RouteConfig { destination: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), nexthop: 10.151.1.100, vlan_id: None, rib_priority: None }], addresses: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: Some(None), bgp_peers: [], autoneg: false, lldp: None, tx_eq: None }], bgp: [], bfd: [] })))
2392024-11-28T01:33:16.547ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3947
2402024-11-28T01:33:16.547ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:81 remote_addr = [fdb0:18c0:4d0d:9fb2::1]:60655
2412024-11-28T01:33:16.722ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:117 peer = [fdb0:18c0:4d0d:9fb2::1]:12346
2422024-11-28T01:33:16.722ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:252 target_sled = [fdb0:18c0:4d0d:9fb2::1]:12346
2432024-11-28T01:33:16.722ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2442024-11-28T01:33:16.722ZINFOSledAgent (RSS): No ledger in /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/rss-service-plan-v5.json file = common/src/ledger.rs:188
2452024-11-28T01:33:16.722ZINFOSledAgent (RSS): No ledger in /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/rss-service-plan-v5.json file = common/src/ledger.rs:188
2462024-11-28T01:33:16.723ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2472024-11-28T01:33:16.723ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2482024-11-28T01:33:16.723ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2492024-11-28T01:33:16.723ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2502024-11-28T01:33:16.727ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:4268 new = [fd00:1122:3344:101::2, ::1] old = [::1]
2512024-11-28T01:33:16.727ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists file = sled-agent/src/services.rs:4284
2522024-11-28T01:33:16.727ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::2, prefix: 64 })) file = illumos-utils/src/running_zone.rs:534 zone = oxz_switch
2532024-11-28T01:33:16.992ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:59456
2542024-11-28T01:33:16.992ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2552024-11-28T01:33:16.992ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2562024-11-28T01:33:16.992ZINFOSledAgent (ServiceManager): No ledger in /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/omicron-zones.json file = common/src/ledger.rs:188
2572024-11-28T01:33:16.992ZINFOSledAgent (ServiceManager): No ledger in /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/omicron-zones.json file = common/src/ledger.rs:188
2582024-11-28T01:33:16.993ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 344 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:59456 req_id = 380b2bad-ef24-4981-943c-b71e23b0af77 response_code = 200 uri = /inventory
2592024-11-28T01:33:17.023ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:38161
2602024-11-28T01:33:17.024ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 42 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:38161 req_id = 849673ef-5b76-4dde-833c-8cfe39ebc9da response_code = 200 uri = /sled-role
2612024-11-28T01:33:17.025ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2622024-11-28T01:33:17.025ZINFOSledAgent (RSS): Writing ledger to /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/.rss-service-plan-v5.json.tmp file = common/src/ledger.rs:199
2632024-11-28T01:33:17.025ZINFOSledAgent (RSS): Writing ledger to /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/.rss-service-plan-v5.json.tmp file = common/src/ledger.rs:199
2642024-11-28T01:33:17.025ZINFOSledAgent (RSS): Service plan written to storage file = sled-agent/src/rack_setup/plan/service.rs:982
2652024-11-28T01:33:17.057ZINFOSledAgent (RSS): attempting to set up sled's storage: OmicronPhysicalDisksConfig { generation: Generation(2), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 722f1c77-2279-4b3f-a5bb-2368dd38f005 (physical_disk), pool_id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: 8281b147-bbe7-4612-aa2b-cccda8ffddd2 (physical_disk), pool_id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 34eea125-29cc-44c9-89f2-418c5aa883d4 (physical_disk), pool_id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 986c02bb-f47f-44f5-862f-fdeecaac5072 (physical_disk), pool_id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 3167525c-7740-41d0-93bc-90d805c5ab2b (physical_disk), pool_id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 6726beb2-4a3c-4be9-b74e-52df61e71615 (physical_disk), pool_id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: f0281ac4-555b-4e41-8e33-f4bcb4074191 (physical_disk), pool_id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: 4f350806-52e3-44ae-89db-c2079a2753bf (physical_disk), pool_id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 7f96bb9b-e434-4260-bb80-b29e4666418a (physical_disk), pool_id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool) }] } file = sled-agent/src/rack_setup/service.rs:369 sled_address = [fd00:1122:3344:101::1]:12345
2662024-11-28T01:33:17.057ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:40608
2672024-11-28T01:33:17.057ZINFOSledAgent: physical disks ensure file = sled-agent/src/sled_agent.rs:896 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
2682024-11-28T01:33:17.057ZINFOSledAgent (StorageManager): Received OmicronPhysicalDisksEnsure { config: OmicronPhysicalDisksConfig { generation: Generation(2), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 722f1c77-2279-4b3f-a5bb-2368dd38f005 (physical_disk), pool_id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: 8281b147-bbe7-4612-aa2b-cccda8ffddd2 (physical_disk), pool_id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 34eea125-29cc-44c9-89f2-418c5aa883d4 (physical_disk), pool_id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 986c02bb-f47f-44f5-862f-fdeecaac5072 (physical_disk), pool_id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 3167525c-7740-41d0-93bc-90d805c5ab2b (physical_disk), pool_id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 6726beb2-4a3c-4be9-b74e-52df61e71615 (physical_disk), pool_id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: f0281ac4-555b-4e41-8e33-f4bcb4074191 (physical_disk), pool_id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: 4f350806-52e3-44ae-89db-c2079a2753bf (physical_disk), pool_id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 7f96bb9b-e434-4260-bb80-b29e4666418a (physical_disk), pool_id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool) }] }, tx: ... } file = sled-storage/src/manager.rs:603
2692024-11-28T01:33:17.057ZINFOSledAgent (StorageManager): No ledger in /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2702024-11-28T01:33:17.057ZINFOSledAgent (StorageManager): No ledger in /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2712024-11-28T01:33:17.057ZINFOSledAgent (StorageManager): No previously-stored 'requested disks', creating new ledger file = sled-storage/src/manager.rs:1171 request = omicron_physical_disks_ensure
2722024-11-28T01:33:17.057ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
2732024-11-28T01:33:17.057ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:342
2742024-11-28T01:33:17.057ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2752024-11-28T01:33:17.057ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } path = /scratch/u2_0.vdev
2762024-11-28T01:33:17.202ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_0.vdev file = sled-hardware/src/disk.rs:367
2772024-11-28T01:33:17.202ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool)
2782024-11-28T01:33:17.439ZINFOSledAgent (VmmReservoirManager): Finished setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:279 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
2792024-11-28T01:33:17.441ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 6865697397444298559 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
2802024-11-28T01:33:17.441ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17855345072000113602 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
2812024-11-28T01:33:17.441ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 7.041378022s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
2822024-11-28T01:33:17.547ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/12368a57-a643-445a-b32d-da10700bd20e) file = sled-agent/src/probe_manager.rs:231 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
2832024-11-28T01:33:17.555ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }
2842024-11-28T01:33:17.564ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:240
2852024-11-28T01:33:17.564ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:242
2862024-11-28T01:33:17.564ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:246
2872024-11-28T01:33:17.564ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:248
2882024-11-28T01:33:17.564ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2892024-11-28T01:33:17.564ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2902024-11-28T01:33:17.564ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
2912024-11-28T01:33:17.601ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2922024-11-28T01:33:17.608ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/zone file = sled-storage/src/dataset.rs:303
2932024-11-28T01:33:17.804ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee file = sled-storage/src/dataset.rs:389
2942024-11-28T01:33:17.811ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:440
2952024-11-28T01:33:17.811ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:357
2962024-11-28T01:33:17.811ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:342
2972024-11-28T01:33:17.811ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2982024-11-28T01:33:17.811ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } path = /scratch/u2_1.vdev
2992024-11-28T01:33:17.822ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_1.vdev file = sled-hardware/src/disk.rs:367
3002024-11-28T01:33:17.822ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 8164ba35-0eff-436a-9b6b-80912419e510 (zpool)
3012024-11-28T01:33:17.937ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }
3022024-11-28T01:33:17.945ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:240
3032024-11-28T01:33:17.945ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:242
3042024-11-28T01:33:17.945ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:246
3052024-11-28T01:33:17.945ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:248
3062024-11-28T01:33:17.945ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
3072024-11-28T01:33:17.945ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
3082024-11-28T01:33:17.945ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3092024-11-28T01:33:17.989ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3102024-11-28T01:33:17.998ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_8164ba35-0eff-436a-9b6b-80912419e510/crypt/zone file = sled-storage/src/dataset.rs:303
3112024-11-28T01:33:18.221ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_8164ba35-0eff-436a-9b6b-80912419e510 file = sled-storage/src/dataset.rs:389
3122024-11-28T01:33:18.229ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:440
3132024-11-28T01:33:18.229ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:357
3142024-11-28T01:33:18.229ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:342
3152024-11-28T01:33:18.229ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3162024-11-28T01:33:18.229ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } path = /scratch/u2_2.vdev
3172024-11-28T01:33:18.239ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_2.vdev file = sled-hardware/src/disk.rs:367
3182024-11-28T01:33:18.239ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool)
3192024-11-28T01:33:18.356ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }
3202024-11-28T01:33:18.365ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:240
3212024-11-28T01:33:18.365ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:242
3222024-11-28T01:33:18.365ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:246
3232024-11-28T01:33:18.365ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:248
3242024-11-28T01:33:18.365ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3252024-11-28T01:33:18.365ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3262024-11-28T01:33:18.365ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3272024-11-28T01:33:18.407ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3282024-11-28T01:33:18.416ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_0176efd8-6a11-46a9-8bcd-a97e447fcb50/crypt/zone file = sled-storage/src/dataset.rs:303
3292024-11-28T01:33:18.550ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/12368a57-a643-445a-b32d-da10700bd20e) file = sled-agent/src/probe_manager.rs:231 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
3302024-11-28T01:33:18.630ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_0176efd8-6a11-46a9-8bcd-a97e447fcb50 file = sled-storage/src/dataset.rs:389
3312024-11-28T01:33:18.638ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:440
3322024-11-28T01:33:18.639ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:357
3332024-11-28T01:33:18.639ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:342
3342024-11-28T01:33:18.639ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3352024-11-28T01:33:18.639ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } path = /scratch/u2_3.vdev
3362024-11-28T01:33:18.650ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_3.vdev file = sled-hardware/src/disk.rs:367
3372024-11-28T01:33:18.650ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = beaaa347-4478-48b4-b888-a5873241ded0 (zpool)
3382024-11-28T01:33:18.767ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }
3392024-11-28T01:33:18.775ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:240
3402024-11-28T01:33:18.775ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:242
3412024-11-28T01:33:18.775ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:246
3422024-11-28T01:33:18.775ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:248
3432024-11-28T01:33:18.775ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3442024-11-28T01:33:18.776ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3452024-11-28T01:33:18.776ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3462024-11-28T01:33:18.818ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3472024-11-28T01:33:18.826ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_beaaa347-4478-48b4-b888-a5873241ded0/crypt/zone file = sled-storage/src/dataset.rs:303
3482024-11-28T01:33:19.029ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_beaaa347-4478-48b4-b888-a5873241ded0 file = sled-storage/src/dataset.rs:389
3492024-11-28T01:33:19.037ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:440
3502024-11-28T01:33:19.037ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:357
3512024-11-28T01:33:19.037ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:342
3522024-11-28T01:33:19.037ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3532024-11-28T01:33:19.037ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } path = /scratch/u2_4.vdev
3542024-11-28T01:33:19.047ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_4.vdev file = sled-hardware/src/disk.rs:367
3552024-11-28T01:33:19.047ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool)
3562024-11-28T01:33:19.164ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }
3572024-11-28T01:33:19.173ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:240
3582024-11-28T01:33:19.173ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:242
3592024-11-28T01:33:19.173ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:246
3602024-11-28T01:33:19.173ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:248
3612024-11-28T01:33:19.173ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3622024-11-28T01:33:19.173ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3632024-11-28T01:33:19.173ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3642024-11-28T01:33:19.216ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3652024-11-28T01:33:19.224ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_216077bb-f8f0-4fbf-8aa0-1db2249e65da/crypt/zone file = sled-storage/src/dataset.rs:303
3662024-11-28T01:33:19.434ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_216077bb-f8f0-4fbf-8aa0-1db2249e65da file = sled-storage/src/dataset.rs:389
3672024-11-28T01:33:19.442ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:440
3682024-11-28T01:33:19.442ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:357
3692024-11-28T01:33:19.442ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:342
3702024-11-28T01:33:19.442ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3712024-11-28T01:33:19.442ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } path = /scratch/u2_5.vdev
3722024-11-28T01:33:19.452ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_5.vdev file = sled-hardware/src/disk.rs:367
3732024-11-28T01:33:19.452ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool)
3742024-11-28T01:33:19.567ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External }
3752024-11-28T01:33:19.574ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:240
3762024-11-28T01:33:19.574ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:242
3772024-11-28T01:33:19.574ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:246
3782024-11-28T01:33:19.574ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:248
3792024-11-28T01:33:19.574ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3802024-11-28T01:33:19.574ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3812024-11-28T01:33:19.574ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3822024-11-28T01:33:19.577ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/12368a57-a643-445a-b32d-da10700bd20e) file = sled-agent/src/probe_manager.rs:231 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
3832024-11-28T01:33:19.613ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3842024-11-28T01:33:19.620ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_977ab5dc-c743-4525-b172-ecf8a9034abe/crypt/zone file = sled-storage/src/dataset.rs:303
3852024-11-28T01:33:19.839ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_977ab5dc-c743-4525-b172-ecf8a9034abe file = sled-storage/src/dataset.rs:389
3862024-11-28T01:33:19.846ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:440
3872024-11-28T01:33:19.846ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:357
3882024-11-28T01:33:19.846ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:342
3892024-11-28T01:33:19.846ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3902024-11-28T01:33:19.846ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } path = /scratch/u2_6.vdev
3912024-11-28T01:33:19.856ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_6.vdev file = sled-hardware/src/disk.rs:367
3922024-11-28T01:33:19.856ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool)
3932024-11-28T01:33:19.969ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }
3942024-11-28T01:33:19.977ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:240
3952024-11-28T01:33:19.977ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:242
3962024-11-28T01:33:19.977ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:246
3972024-11-28T01:33:19.977ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:248
3982024-11-28T01:33:19.978ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3992024-11-28T01:33:19.978ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
4002024-11-28T01:33:19.978ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
4012024-11-28T01:33:20.018ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4022024-11-28T01:33:20.026ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_2fcc59ee-335a-4c80-bbcf-e66263237ec3/crypt/zone file = sled-storage/src/dataset.rs:303
4032024-11-28T01:33:20.236ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_2fcc59ee-335a-4c80-bbcf-e66263237ec3 file = sled-storage/src/dataset.rs:389
4042024-11-28T01:33:20.244ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:440
4052024-11-28T01:33:20.244ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:357
4062024-11-28T01:33:20.244ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:342
4072024-11-28T01:33:20.244ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4082024-11-28T01:33:20.244ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } path = /scratch/u2_7.vdev
4092024-11-28T01:33:20.255ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_7.vdev file = sled-hardware/src/disk.rs:367
4102024-11-28T01:33:20.255ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool)
4112024-11-28T01:33:20.373ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }
4122024-11-28T01:33:20.381ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:240
4132024-11-28T01:33:20.381ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:242
4142024-11-28T01:33:20.381ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:246
4152024-11-28T01:33:20.381ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:248
4162024-11-28T01:33:20.381ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4172024-11-28T01:33:20.381ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4182024-11-28T01:33:20.381ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
4192024-11-28T01:33:20.428ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4202024-11-28T01:33:20.436ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_c7ca3a79-57f5-4e47-8807-03bd67966814/crypt/zone file = sled-storage/src/dataset.rs:303
4212024-11-28T01:33:20.580ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/12368a57-a643-445a-b32d-da10700bd20e) file = sled-agent/src/probe_manager.rs:231 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
4222024-11-28T01:33:20.648ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_c7ca3a79-57f5-4e47-8807-03bd67966814 file = sled-storage/src/dataset.rs:389
4232024-11-28T01:33:20.655ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:440
4242024-11-28T01:33:20.655ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:357
4252024-11-28T01:33:20.655ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:342
4262024-11-28T01:33:20.655ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4272024-11-28T01:33:20.655ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } path = /scratch/u2_8.vdev
4282024-11-28T01:33:20.666ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_8.vdev file = sled-hardware/src/disk.rs:367
4292024-11-28T01:33:20.666ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool)
4302024-11-28T01:33:20.788ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }
4312024-11-28T01:33:20.796ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:240
4322024-11-28T01:33:20.796ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:242
4332024-11-28T01:33:20.796ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:246
4342024-11-28T01:33:20.796ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:248
4352024-11-28T01:33:20.796ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4362024-11-28T01:33:20.796ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4372024-11-28T01:33:20.796ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
4382024-11-28T01:33:20.835ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4392024-11-28T01:33:20.843ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_75598bf6-d28b-4334-94db-398fc2a8b539/crypt/zone file = sled-storage/src/dataset.rs:303
4402024-11-28T01:33:21.052ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_75598bf6-d28b-4334-94db-398fc2a8b539 file = sled-storage/src/dataset.rs:389
4412024-11-28T01:33:21.060ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:440
4422024-11-28T01:33:21.060ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:357
4432024-11-28T01:33:21.060ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4442024-11-28T01:33:21.060ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(2), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 2d6a91dc-89a8-488e-9efc-6f5bec9b10d5 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 7f63c436-f542-43a4-9af6-de72a4c8391f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
4452024-11-28T01:33:21.060ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
4462024-11-28T01:33:21.060ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
4472024-11-28T01:33:21.061ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4482024-11-28T01:33:21.061ZINFOSledAgent: physical disks ensure: Updated storage file = sled-agent/src/sled_agent.rs:900 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
4492024-11-28T01:33:21.061ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
4502024-11-28T01:33:21.061ZINFOSledAgent: physical disks ensure: Propagating new generation of disks file = sled-agent/src/sled_agent.rs:926 generation = Generation(2) sled_id = 12368a57-a643-445a-b32d-da10700bd20e
4512024-11-28T01:33:21.061ZINFOSledAgent: physical disks ensure: Updated storage monitor file = sled-agent/src/sled_agent.rs:931 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
4522024-11-28T01:33:21.061ZINFOSledAgent: physical disks ensure: Updated zone bundler file = sled-agent/src/sled_agent.rs:936 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
4532024-11-28T01:33:21.061ZINFOSledAgent (ProbeManager): use_only_these_disks: Processing new request file = sled-agent/src/probe_manager.rs:122 gen = Generation(2) sled_id = 12368a57-a643-445a-b32d-da10700bd20e
4542024-11-28T01:33:21.061ZINFOSledAgent: physical disks ensure: Updated probes file = sled-agent/src/sled_agent.rs:942 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
4552024-11-28T01:33:21.061ZINFOSledAgent (InstanceManager): use_only_these_disks: Processing new request file = sled-agent/src/instance_manager.rs:799 gen = Generation(2)
4562024-11-28T01:33:21.061ZINFOSledAgent: physical disks ensure: Updated instances file = sled-agent/src/sled_agent.rs:947 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
4572024-11-28T01:33:21.061ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4003795 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:40608 req_id = 45aff9c8-b989-4a69-891f-414fdfa4f86b response_code = 200 uri = /omicron-physical-disks
4582024-11-28T01:33:21.094ZINFOSledAgent (RSS): attempting to set up sled's datasets: DatasetsConfig { generation: Generation(1), datasets: {02eac503-c88f-4fa5-8ae7-bbe6fb12dad0 (dataset): DatasetConfig { id: 02eac503-c88f-4fa5-8ae7-bbe6fb12dad0 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 06426023-6f99-4e51-a906-410b99fc3a61 (dataset): DatasetConfig { id: 06426023-6f99-4e51-a906-410b99fc3a61 (dataset), name: DatasetName { pool_name: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }, kind: TransientZone { name: "oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 06c71eed-74b5-4499-8219-899efdaa310e (dataset): DatasetConfig { id: 06c71eed-74b5-4499-8219-899efdaa310e (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 084c630b-125e-4dca-bafe-1db2ecc6c424 (dataset): DatasetConfig { id: 084c630b-125e-4dca-bafe-1db2ecc6c424 (dataset), name: DatasetName { pool_name: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0d00a313-5015-4c90-b157-d4c24ea9312f (dataset): DatasetConfig { id: 0d00a313-5015-4c90-b157-d4c24ea9312f (dataset), name: DatasetName { pool_name: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1348e161-2372-4e40-b5b7-f45eb06fd71f (dataset): DatasetConfig { id: 1348e161-2372-4e40-b5b7-f45eb06fd71f (dataset), name: DatasetName { pool_name: ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 18f812f3-3f9e-48e2-aedc-15c71f6394b3 (dataset): DatasetConfig { id: 18f812f3-3f9e-48e2-aedc-15c71f6394b3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1a26e525-760b-4223-8f0d-e135067f5ab2 (dataset): DatasetConfig { id: 1a26e525-760b-4223-8f0d-e135067f5ab2 (dataset), name: DatasetName { pool_name: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 1ab6f113-7eb1-4258-b869-fb8722ccf018 (dataset): DatasetConfig { id: 1ab6f113-7eb1-4258-b869-fb8722ccf018 (dataset), name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1c19cbe7-b2e7-433b-8a05-28ab38462de4 (dataset): DatasetConfig { id: 1c19cbe7-b2e7-433b-8a05-28ab38462de4 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1c7226e0-a75e-462e-97eb-a9c91f80219b (dataset): DatasetConfig { id: 1c7226e0-a75e-462e-97eb-a9c91f80219b (dataset), name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 1f484e44-8e1c-40cb-88c6-60a2242f3c24 (dataset): DatasetConfig { id: 1f484e44-8e1c-40cb-88c6-60a2242f3c24 (dataset), name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2011bb8e-a55c-464a-bbd2-58b0b68f33d4 (dataset): DatasetConfig { id: 2011bb8e-a55c-464a-bbd2-58b0b68f33d4 (dataset), name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2159cd87-5d28-48ba-aff1-962c417d20ef (dataset): DatasetConfig { id: 2159cd87-5d28-48ba-aff1-962c417d20ef (dataset), name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 225d1f2f-4482-47bb-ad7c-6d4d85d8f9dd (dataset): DatasetConfig { id: 225d1f2f-4482-47bb-ad7c-6d4d85d8f9dd (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: TransientZone { name: "oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2cb29ef5-9bc3-4feb-b7ee-a98c4ce552da (dataset): DatasetConfig { id: 2cb29ef5-9bc3-4feb-b7ee-a98c4ce552da (dataset), name: DatasetName { pool_name: ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2dec88d1-0d82-4b6a-9bb8-19b029a72b7f (dataset): DatasetConfig { id: 2dec88d1-0d82-4b6a-9bb8-19b029a72b7f (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2e058a3c-de3b-43b9-93b3-7fc38ebac3c0 (dataset): DatasetConfig { id: 2e058a3c-de3b-43b9-93b3-7fc38ebac3c0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 304f14f2-11fd-4cc7-9f74-29c0fa6f9fae (dataset): DatasetConfig { id: 304f14f2-11fd-4cc7-9f74-29c0fa6f9fae (dataset), name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 36f41d98-a7ec-4332-83d5-7cb82b4dd38b (dataset): DatasetConfig { id: 36f41d98-a7ec-4332-83d5-7cb82b4dd38b (dataset), name: DatasetName { pool_name: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 37b8113c-2568-4e76-9757-a324da64d0f0 (dataset): DatasetConfig { id: 37b8113c-2568-4e76-9757-a324da64d0f0 (dataset), name: DatasetName { pool_name: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3d9be209-a643-443a-975c-8a426acc2323 (dataset): DatasetConfig { id: 3d9be209-a643-443a-975c-8a426acc2323 (dataset), name: DatasetName { pool_name: ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3e4a729a-c3a9-4c58-b305-ec70b65662bf (dataset): DatasetConfig { id: 3e4a729a-c3a9-4c58-b305-ec70b65662bf (dataset), name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 42bcba77-5acb-4b7e-a7b4-df2cfe48bcb0 (dataset): DatasetConfig { id: 42bcba77-5acb-4b7e-a7b4-df2cfe48bcb0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 44437359-22f8-43a1-ba80-07bdcecd9c69 (dataset): DatasetConfig { id: 44437359-22f8-43a1-ba80-07bdcecd9c69 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4897b838-a0de-4e8f-8d84-4c6b4f43138a (dataset): DatasetConfig { id: 4897b838-a0de-4e8f-8d84-4c6b4f43138a (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5160c37e-5e80-42fa-8503-c71a39bcc42e (dataset): DatasetConfig { id: 5160c37e-5e80-42fa-8503-c71a39bcc42e (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 52811a87-db12-40ae-905c-de912e79b720 (dataset): DatasetConfig { id: 52811a87-db12-40ae-905c-de912e79b720 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 549e9727-2ed4-4a92-914e-ce1c3a258a7e (dataset): DatasetConfig { id: 549e9727-2ed4-4a92-914e-ce1c3a258a7e (dataset), name: DatasetName { pool_name: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5c130012-c0f8-4ed8-8bc5-4e10b7097182 (dataset): DatasetConfig { id: 5c130012-c0f8-4ed8-8bc5-4e10b7097182 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6c6bfc49-c36a-40d1-a842-6421d7ea3122 (dataset): DatasetConfig { id: 6c6bfc49-c36a-40d1-a842-6421d7ea3122 (dataset), name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6f41784d-d6b9-49f2-936a-171c858f785f (dataset): DatasetConfig { id: 6f41784d-d6b9-49f2-936a-171c858f785f (dataset), name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 7dceccdd-aaf7-4ad9-89ba-b7235f6d3d19 (dataset): DatasetConfig { id: 7dceccdd-aaf7-4ad9-89ba-b7235f6d3d19 (dataset), name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 7ee98ba5-74c7-44e7-9223-34a438dfb5aa (dataset): DatasetConfig { id: 7ee98ba5-74c7-44e7-9223-34a438dfb5aa (dataset), name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 801622b1-53e0-44d0-ac83-d241609bf575 (dataset): DatasetConfig { id: 801622b1-53e0-44d0-ac83-d241609bf575 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 84f3cff9-0cdf-45c3-9524-841fd189a7d4 (dataset): DatasetConfig { id: 84f3cff9-0cdf-45c3-9524-841fd189a7d4 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 87527362-3759-4b16-9406-d9359aa2e63f (dataset): DatasetConfig { id: 87527362-3759-4b16-9406-d9359aa2e63f (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Clickhouse }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9079c551-3296-4b75-b1c3-11a2330a5412 (dataset): DatasetConfig { id: 9079c551-3296-4b75-b1c3-11a2330a5412 (dataset), name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 930f38e6-e7e4-4827-92ec-c3db7553d032 (dataset): DatasetConfig { id: 930f38e6-e7e4-4827-92ec-c3db7553d032 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 99831272-1101-4b4b-b49f-4259675120fa (dataset): DatasetConfig { id: 99831272-1101-4b4b-b49f-4259675120fa (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a1090228-963f-46fa-98de-1fb9dfd6bf2c (dataset): DatasetConfig { id: a1090228-963f-46fa-98de-1fb9dfd6bf2c (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a269c129-6016-4dec-97f7-f91d98de6d90 (dataset): DatasetConfig { id: a269c129-6016-4dec-97f7-f91d98de6d90 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a32739b5-809b-4eb6-b098-d4289587d779 (dataset): DatasetConfig { id: a32739b5-809b-4eb6-b098-d4289587d779 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a4cf1f1a-918a-48db-a50e-5ee32c31001e (dataset): DatasetConfig { id: a4cf1f1a-918a-48db-a50e-5ee32c31001e (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a4ec692f-7fe9-4b5f-8f5f-20c463e56866 (dataset): DatasetConfig { id: a4ec692f-7fe9-4b5f-8f5f-20c463e56866 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a5abc30a-1c8e-41a5-b38b-89206467b95f (dataset): DatasetConfig { id: a5abc30a-1c8e-41a5-b38b-89206467b95f (dataset), name: DatasetName { pool_name: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a96ad8fd-b76b-4976-956f-14502568df2b (dataset): DatasetConfig { id: a96ad8fd-b76b-4976-956f-14502568df2b (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, adab569d-c45d-4186-87e3-ecc1c9ccd50b (dataset): DatasetConfig { id: adab569d-c45d-4186-87e3-ecc1c9ccd50b (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, b65fc875-8a47-4a32-a24a-825b80e7527e (dataset): DatasetConfig { id: b65fc875-8a47-4a32-a24a-825b80e7527e (dataset), name: DatasetName { pool_name: ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, b853b2bd-4651-487f-bf6d-22be3316d37a (dataset): DatasetConfig { id: b853b2bd-4651-487f-bf6d-22be3316d37a (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, c3166a1d-ab02-49b3-9d6e-8bebba18f22a (dataset): DatasetConfig { id: c3166a1d-ab02-49b3-9d6e-8bebba18f22a (dataset), name: DatasetName { pool_name: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c46624b1-0060-4830-b376-21a8fe22b435 (dataset): DatasetConfig { id: c46624b1-0060-4830-b376-21a8fe22b435 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c9795634-88f4-49a9-b451-65f9a64ef57a (dataset): DatasetConfig { id: c9795634-88f4-49a9-b451-65f9a64ef57a (dataset), name: DatasetName { pool_name: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, kind: TransientZone { name: "oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, cd172b57-1618-4ecd-b72f-b8d8fe6aec66 (dataset): DatasetConfig { id: cd172b57-1618-4ecd-b72f-b8d8fe6aec66 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, cf48fd7d-0cf9-4e47-bb7b-2f2f5d09c0e4 (dataset): DatasetConfig { id: cf48fd7d-0cf9-4e47-bb7b-2f2f5d09c0e4 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, cf9a6a88-1653-4eef-b2a6-1c6f743cd6d8 (dataset): DatasetConfig { id: cf9a6a88-1653-4eef-b2a6-1c6f743cd6d8 (dataset), name: DatasetName { pool_name: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, da45ad03-97d0-4637-86da-07b941e98aa4 (dataset): DatasetConfig { id: da45ad03-97d0-4637-86da-07b941e98aa4 (dataset), name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, db0c548d-dc39-4172-bab0-1a4c1631d366 (dataset): DatasetConfig { id: db0c548d-dc39-4172-bab0-1a4c1631d366 (dataset), name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, dceb8fe3-4a25-47a0-8deb-a487401bd440 (dataset): DatasetConfig { id: dceb8fe3-4a25-47a0-8deb-a487401bd440 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e1970908-ed20-477a-a4bf-036c948e306c (dataset): DatasetConfig { id: e1970908-ed20-477a-a4bf-036c948e306c (dataset), name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, e2d4e114-4c76-4b22-98ec-1f184d2c2fa5 (dataset): DatasetConfig { id: e2d4e114-4c76-4b22-98ec-1f184d2c2fa5 (dataset), name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e3dd1637-9032-4921-b854-764bba135f27 (dataset): DatasetConfig { id: e3dd1637-9032-4921-b854-764bba135f27 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e56de823-d8bc-48ac-944a-0b0e6134f526 (dataset): DatasetConfig { id: e56de823-d8bc-48ac-944a-0b0e6134f526 (dataset), name: DatasetName { pool_name: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e736ed56-bcd4-4664-bc9f-f3d42ef0d618 (dataset): DatasetConfig { id: e736ed56-bcd4-4664-bc9f-f3d42ef0d618 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f59d1bcc-9e13-4e4b-8f26-eb5c8d2a26ed (dataset): DatasetConfig { id: f59d1bcc-9e13-4e4b-8f26-eb5c8d2a26ed (dataset), name: DatasetName { pool_name: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, fa2f7273-7621-40d5-99a0-a7eba43ece0c (dataset): DatasetConfig { id: fa2f7273-7621-40d5-99a0-a7eba43ece0c (dataset), name: DatasetName { pool_name: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }} } file = sled-agent/src/rack_setup/service.rs:444 sled_address = [fd00:1122:3344:101::1]:12345
4592024-11-28T01:33:21.094ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:61855
4602024-11-28T01:33:21.094ZINFOSledAgent: datasets ensure file = sled-agent/src/sled_agent.rs:862 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
4612024-11-28T01:33:21.094ZINFOSledAgent (StorageManager): Received DatasetsEnsure { config: DatasetsConfig { generation: Generation(1), datasets: {02eac503-c88f-4fa5-8ae7-bbe6fb12dad0 (dataset): DatasetConfig { id: 02eac503-c88f-4fa5-8ae7-bbe6fb12dad0 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 06426023-6f99-4e51-a906-410b99fc3a61 (dataset): DatasetConfig { id: 06426023-6f99-4e51-a906-410b99fc3a61 (dataset), name: DatasetName { pool_name: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }, kind: TransientZone { name: "oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 06c71eed-74b5-4499-8219-899efdaa310e (dataset): DatasetConfig { id: 06c71eed-74b5-4499-8219-899efdaa310e (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 084c630b-125e-4dca-bafe-1db2ecc6c424 (dataset): DatasetConfig { id: 084c630b-125e-4dca-bafe-1db2ecc6c424 (dataset), name: DatasetName { pool_name: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0d00a313-5015-4c90-b157-d4c24ea9312f (dataset): DatasetConfig { id: 0d00a313-5015-4c90-b157-d4c24ea9312f (dataset), name: DatasetName { pool_name: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1348e161-2372-4e40-b5b7-f45eb06fd71f (dataset): DatasetConfig { id: 1348e161-2372-4e40-b5b7-f45eb06fd71f (dataset), name: DatasetName { pool_name: ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 18f812f3-3f9e-48e2-aedc-15c71f6394b3 (dataset): DatasetConfig { id: 18f812f3-3f9e-48e2-aedc-15c71f6394b3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1a26e525-760b-4223-8f0d-e135067f5ab2 (dataset): DatasetConfig { id: 1a26e525-760b-4223-8f0d-e135067f5ab2 (dataset), name: DatasetName { pool_name: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 1ab6f113-7eb1-4258-b869-fb8722ccf018 (dataset): DatasetConfig { id: 1ab6f113-7eb1-4258-b869-fb8722ccf018 (dataset), name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1c19cbe7-b2e7-433b-8a05-28ab38462de4 (dataset): DatasetConfig { id: 1c19cbe7-b2e7-433b-8a05-28ab38462de4 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1c7226e0-a75e-462e-97eb-a9c91f80219b (dataset): DatasetConfig { id: 1c7226e0-a75e-462e-97eb-a9c91f80219b (dataset), name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 1f484e44-8e1c-40cb-88c6-60a2242f3c24 (dataset): DatasetConfig { id: 1f484e44-8e1c-40cb-88c6-60a2242f3c24 (dataset), name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2011bb8e-a55c-464a-bbd2-58b0b68f33d4 (dataset): DatasetConfig { id: 2011bb8e-a55c-464a-bbd2-58b0b68f33d4 (dataset), name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2159cd87-5d28-48ba-aff1-962c417d20ef (dataset): DatasetConfig { id: 2159cd87-5d28-48ba-aff1-962c417d20ef (dataset), name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 225d1f2f-4482-47bb-ad7c-6d4d85d8f9dd (dataset): DatasetConfig { id: 225d1f2f-4482-47bb-ad7c-6d4d85d8f9dd (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: TransientZone { name: "oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2cb29ef5-9bc3-4feb-b7ee-a98c4ce552da (dataset): DatasetConfig { id: 2cb29ef5-9bc3-4feb-b7ee-a98c4ce552da (dataset), name: DatasetName { pool_name: ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2dec88d1-0d82-4b6a-9bb8-19b029a72b7f (dataset): DatasetConfig { id: 2dec88d1-0d82-4b6a-9bb8-19b029a72b7f (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2e058a3c-de3b-43b9-93b3-7fc38ebac3c0 (dataset): DatasetConfig { id: 2e058a3c-de3b-43b9-93b3-7fc38ebac3c0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 304f14f2-11fd-4cc7-9f74-29c0fa6f9fae (dataset): DatasetConfig { id: 304f14f2-11fd-4cc7-9f74-29c0fa6f9fae (dataset), name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 36f41d98-a7ec-4332-83d5-7cb82b4dd38b (dataset): DatasetConfig { id: 36f41d98-a7ec-4332-83d5-7cb82b4dd38b (dataset), name: DatasetName { pool_name: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 37b8113c-2568-4e76-9757-a324da64d0f0 (dataset): DatasetConfig { id: 37b8113c-2568-4e76-9757-a324da64d0f0 (dataset), name: DatasetName { pool_name: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3d9be209-a643-443a-975c-8a426acc2323 (dataset): DatasetConfig { id: 3d9be209-a643-443a-975c-8a426acc2323 (dataset), name: DatasetName { pool_name: ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3e4a729a-c3a9-4c58-b305-ec70b65662bf (dataset): DatasetConfig { id: 3e4a729a-c3a9-4c58-b305-ec70b65662bf (dataset), name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 42bcba77-5acb-4b7e-a7b4-df2cfe48bcb0 (dataset): DatasetConfig { id: 42bcba77-5acb-4b7e-a7b4-df2cfe48bcb0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 44437359-22f8-43a1-ba80-07bdcecd9c69 (dataset): DatasetConfig { id: 44437359-22f8-43a1-ba80-07bdcecd9c69 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4897b838-a0de-4e8f-8d84-4c6b4f43138a (dataset): DatasetConfig { id: 4897b838-a0de-4e8f-8d84-4c6b4f43138a (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5160c37e-5e80-42fa-8503-c71a39bcc42e (dataset): DatasetConfig { id: 5160c37e-5e80-42fa-8503-c71a39bcc42e (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 52811a87-db12-40ae-905c-de912e79b720 (dataset): DatasetConfig { id: 52811a87-db12-40ae-905c-de912e79b720 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 549e9727-2ed4-4a92-914e-ce1c3a258a7e (dataset): DatasetConfig { id: 549e9727-2ed4-4a92-914e-ce1c3a258a7e (dataset), name: DatasetName { pool_name: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5c130012-c0f8-4ed8-8bc5-4e10b7097182 (dataset): DatasetConfig { id: 5c130012-c0f8-4ed8-8bc5-4e10b7097182 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6c6bfc49-c36a-40d1-a842-6421d7ea3122 (dataset): DatasetConfig { id: 6c6bfc49-c36a-40d1-a842-6421d7ea3122 (dataset), name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6f41784d-d6b9-49f2-936a-171c858f785f (dataset): DatasetConfig { id: 6f41784d-d6b9-49f2-936a-171c858f785f (dataset), name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 7dceccdd-aaf7-4ad9-89ba-b7235f6d3d19 (dataset): DatasetConfig { id: 7dceccdd-aaf7-4ad9-89ba-b7235f6d3d19 (dataset), name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 7ee98ba5-74c7-44e7-9223-34a438dfb5aa (dataset): DatasetConfig { id: 7ee98ba5-74c7-44e7-9223-34a438dfb5aa (dataset), name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 801622b1-53e0-44d0-ac83-d241609bf575 (dataset): DatasetConfig { id: 801622b1-53e0-44d0-ac83-d241609bf575 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 84f3cff9-0cdf-45c3-9524-841fd189a7d4 (dataset): DatasetConfig { id: 84f3cff9-0cdf-45c3-9524-841fd189a7d4 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 87527362-3759-4b16-9406-d9359aa2e63f (dataset): DatasetConfig { id: 87527362-3759-4b16-9406-d9359aa2e63f (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Clickhouse }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9079c551-3296-4b75-b1c3-11a2330a5412 (dataset): DatasetConfig { id: 9079c551-3296-4b75-b1c3-11a2330a5412 (dataset), name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 930f38e6-e7e4-4827-92ec-c3db7553d032 (dataset): DatasetConfig { id: 930f38e6-e7e4-4827-92ec-c3db7553d032 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 99831272-1101-4b4b-b49f-4259675120fa (dataset): DatasetConfig { id: 99831272-1101-4b4b-b49f-4259675120fa (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a1090228-963f-46fa-98de-1fb9dfd6bf2c (dataset): DatasetConfig { id: a1090228-963f-46fa-98de-1fb9dfd6bf2c (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a269c129-6016-4dec-97f7-f91d98de6d90 (dataset): DatasetConfig { id: a269c129-6016-4dec-97f7-f91d98de6d90 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a32739b5-809b-4eb6-b098-d4289587d779 (dataset): DatasetConfig { id: a32739b5-809b-4eb6-b098-d4289587d779 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a4cf1f1a-918a-48db-a50e-5ee32c31001e (dataset): DatasetConfig { id: a4cf1f1a-918a-48db-a50e-5ee32c31001e (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a4ec692f-7fe9-4b5f-8f5f-20c463e56866 (dataset): DatasetConfig { id: a4ec692f-7fe9-4b5f-8f5f-20c463e56866 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a5abc30a-1c8e-41a5-b38b-89206467b95f (dataset): DatasetConfig { id: a5abc30a-1c8e-41a5-b38b-89206467b95f (dataset), name: DatasetName { pool_name: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a96ad8fd-b76b-4976-956f-14502568df2b (dataset): DatasetConfig { id: a96ad8fd-b76b-4976-956f-14502568df2b (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, adab569d-c45d-4186-87e3-ecc1c9ccd50b (dataset): DatasetConfig { id: adab569d-c45d-4186-87e3-ecc1c9ccd50b (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, b65fc875-8a47-4a32-a24a-825b80e7527e (dataset): DatasetConfig { id: b65fc875-8a47-4a32-a24a-825b80e7527e (dataset), name: DatasetName { pool_name: ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, b853b2bd-4651-487f-bf6d-22be3316d37a (dataset): DatasetConfig { id: b853b2bd-4651-487f-bf6d-22be3316d37a (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, c3166a1d-ab02-49b3-9d6e-8bebba18f22a (dataset): DatasetConfig { id: c3166a1d-ab02-49b3-9d6e-8bebba18f22a (dataset), name: DatasetName { pool_name: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c46624b1-0060-4830-b376-21a8fe22b435 (dataset): DatasetConfig { id: c46624b1-0060-4830-b376-21a8fe22b435 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c9795634-88f4-49a9-b451-65f9a64ef57a (dataset): DatasetConfig { id: c9795634-88f4-49a9-b451-65f9a64ef57a (dataset), name: DatasetName { pool_name: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, kind: TransientZone { name: "oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, cd172b57-1618-4ecd-b72f-b8d8fe6aec66 (dataset): DatasetConfig { id: cd172b57-1618-4ecd-b72f-b8d8fe6aec66 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, cf48fd7d-0cf9-4e47-bb7b-2f2f5d09c0e4 (dataset): DatasetConfig { id: cf48fd7d-0cf9-4e47-bb7b-2f2f5d09c0e4 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, cf9a6a88-1653-4eef-b2a6-1c6f743cd6d8 (dataset): DatasetConfig { id: cf9a6a88-1653-4eef-b2a6-1c6f743cd6d8 (dataset), name: DatasetName { pool_name: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, da45ad03-97d0-4637-86da-07b941e98aa4 (dataset): DatasetConfig { id: da45ad03-97d0-4637-86da-07b941e98aa4 (dataset), name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, db0c548d-dc39-4172-bab0-1a4c1631d366 (dataset): DatasetConfig { id: db0c548d-dc39-4172-bab0-1a4c1631d366 (dataset), name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, dceb8fe3-4a25-47a0-8deb-a487401bd440 (dataset): DatasetConfig { id: dceb8fe3-4a25-47a0-8deb-a487401bd440 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e1970908-ed20-477a-a4bf-036c948e306c (dataset): DatasetConfig { id: e1970908-ed20-477a-a4bf-036c948e306c (dataset), name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, e2d4e114-4c76-4b22-98ec-1f184d2c2fa5 (dataset): DatasetConfig { id: e2d4e114-4c76-4b22-98ec-1f184d2c2fa5 (dataset), name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e3dd1637-9032-4921-b854-764bba135f27 (dataset): DatasetConfig { id: e3dd1637-9032-4921-b854-764bba135f27 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e56de823-d8bc-48ac-944a-0b0e6134f526 (dataset): DatasetConfig { id: e56de823-d8bc-48ac-944a-0b0e6134f526 (dataset), name: DatasetName { pool_name: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e736ed56-bcd4-4664-bc9f-f3d42ef0d618 (dataset): DatasetConfig { id: e736ed56-bcd4-4664-bc9f-f3d42ef0d618 (dataset), name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f59d1bcc-9e13-4e4b-8f26-eb5c8d2a26ed (dataset): DatasetConfig { id: f59d1bcc-9e13-4e4b-8f26-eb5c8d2a26ed (dataset), name: DatasetName { pool_name: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, fa2f7273-7621-40d5-99a0-a7eba43ece0c (dataset): DatasetConfig { id: fa2f7273-7621-40d5-99a0-a7eba43ece0c (dataset), name: DatasetName { pool_name: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }} }, tx: ... } file = sled-storage/src/manager.rs:603
4622024-11-28T01:33:21.094ZINFOSledAgent (StorageManager): No ledger in /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/omicron-datasets.json file = common/src/ledger.rs:188 request = datasets_ensure
4632024-11-28T01:33:21.094ZINFOSledAgent (StorageManager): No ledger in /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/omicron-datasets.json file = common/src/ledger.rs:188 request = datasets_ensure
4642024-11-28T01:33:21.094ZINFOSledAgent (StorageManager): No previously-stored 'requested datasets', creating new ledger file = sled-storage/src/manager.rs:905 request = datasets_ensure
4652024-11-28T01:33:21.094ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4662024-11-28T01:33:21.094ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/zone/oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7"), full_name: "oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/zone/oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7" } file = sled-storage/src/manager.rs:1331
4672024-11-28T01:33:21.214ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4682024-11-28T01:33:21.214ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/75598bf6-d28b-4334-94db-398fc2a8b539/crypt/zone/oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229"), full_name: "oxp_75598bf6-d28b-4334-94db-398fc2a8b539/crypt/zone/oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229" } file = sled-storage/src/manager.rs:1331
4692024-11-28T01:33:21.333ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4702024-11-28T01:33:21.333ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_8164ba35-0eff-436a-9b6b-80912419e510/crypt/external_dns" } file = sled-storage/src/manager.rs:1331
4712024-11-28T01:33:21.446ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4722024-11-28T01:33:21.446ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/75598bf6-d28b-4334-94db-398fc2a8b539/crypt/zone"), full_name: "oxp_75598bf6-d28b-4334-94db-398fc2a8b539/crypt/zone" } file = sled-storage/src/manager.rs:1331
4732024-11-28T01:33:21.529ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4742024-11-28T01:33:21.529ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_c7ca3a79-57f5-4e47-8807-03bd67966814/crucible" } file = sled-storage/src/manager.rs:1331
4752024-11-28T01:33:21.585ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/12368a57-a643-445a-b32d-da10700bd20e) file = sled-agent/src/probe_manager.rs:231 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
4762024-11-28T01:33:21.642ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4772024-11-28T01:33:21.642ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/977ab5dc-c743-4525-b172-ecf8a9034abe/crypt/zone/oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104"), full_name: "oxp_977ab5dc-c743-4525-b172-ecf8a9034abe/crypt/zone/oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104" } file = sled-storage/src/manager.rs:1331
4782024-11-28T01:33:21.763ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4792024-11-28T01:33:21.763ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/2fcc59ee-335a-4c80-bbcf-e66263237ec3/crypt/zone/oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d"), full_name: "oxp_2fcc59ee-335a-4c80-bbcf-e66263237ec3/crypt/zone/oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d" } file = sled-storage/src/manager.rs:1331
4802024-11-28T01:33:21.883ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4812024-11-28T01:33:21.883ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/2fcc59ee-335a-4c80-bbcf-e66263237ec3/crypt/debug"), full_name: "oxp_2fcc59ee-335a-4c80-bbcf-e66263237ec3/crypt/debug" } file = sled-storage/src/manager.rs:1331
4822024-11-28T01:33:21.967ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4832024-11-28T01:33:21.967ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/beaaa347-4478-48b4-b888-a5873241ded0/crypt/zone/oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567"), full_name: "oxp_beaaa347-4478-48b4-b888-a5873241ded0/crypt/zone/oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567" } file = sled-storage/src/manager.rs:1331
4842024-11-28T01:33:22.087ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4852024-11-28T01:33:22.087ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0176efd8-6a11-46a9-8bcd-a97e447fcb50/crypt/zone/oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08"), full_name: "oxp_0176efd8-6a11-46a9-8bcd-a97e447fcb50/crypt/zone/oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08" } file = sled-storage/src/manager.rs:1331
4862024-11-28T01:33:22.208ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4872024-11-28T01:33:22.208ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/beaaa347-4478-48b4-b888-a5873241ded0/crypt/debug"), full_name: "oxp_beaaa347-4478-48b4-b888-a5873241ded0/crypt/debug" } file = sled-storage/src/manager.rs:1331
4882024-11-28T01:33:22.292ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4892024-11-28T01:33:22.292ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/beaaa347-4478-48b4-b888-a5873241ded0/crypt/zone/oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28"), full_name: "oxp_beaaa347-4478-48b4-b888-a5873241ded0/crypt/zone/oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28" } file = sled-storage/src/manager.rs:1331
4902024-11-28T01:33:22.414ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4912024-11-28T01:33:22.414ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_216077bb-f8f0-4fbf-8aa0-1db2249e65da/crucible" } file = sled-storage/src/manager.rs:1331
4922024-11-28T01:33:22.523ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4932024-11-28T01:33:22.523ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/beaaa347-4478-48b4-b888-a5873241ded0/crypt/zone/oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea"), full_name: "oxp_beaaa347-4478-48b4-b888-a5873241ded0/crypt/zone/oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea" } file = sled-storage/src/manager.rs:1331
4942024-11-28T01:33:22.587ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/12368a57-a643-445a-b32d-da10700bd20e) file = sled-agent/src/probe_manager.rs:231 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
4952024-11-28T01:33:22.644ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4962024-11-28T01:33:22.644ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/zone/oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b"), full_name: "oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/zone/oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b" } file = sled-storage/src/manager.rs:1331
4972024-11-28T01:33:22.761ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4982024-11-28T01:33:22.761ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/977ab5dc-c743-4525-b172-ecf8a9034abe/crypt/zone"), full_name: "oxp_977ab5dc-c743-4525-b172-ecf8a9034abe/crypt/zone" } file = sled-storage/src/manager.rs:1331
4992024-11-28T01:33:22.844ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5002024-11-28T01:33:22.844ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8164ba35-0eff-436a-9b6b-80912419e510/crypt/zone/oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf"), full_name: "oxp_8164ba35-0eff-436a-9b6b-80912419e510/crypt/zone/oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf" } file = sled-storage/src/manager.rs:1331
5012024-11-28T01:33:22.964ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5022024-11-28T01:33:22.964ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_0176efd8-6a11-46a9-8bcd-a97e447fcb50/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
5032024-11-28T01:33:23.080ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5042024-11-28T01:33:23.080ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/beaaa347-4478-48b4-b888-a5873241ded0/crypt/zone"), full_name: "oxp_beaaa347-4478-48b4-b888-a5873241ded0/crypt/zone" } file = sled-storage/src/manager.rs:1331
5052024-11-28T01:33:23.163ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5062024-11-28T01:33:23.163ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/75598bf6-d28b-4334-94db-398fc2a8b539/crypt/zone/oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c"), full_name: "oxp_75598bf6-d28b-4334-94db-398fc2a8b539/crypt/zone/oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c" } file = sled-storage/src/manager.rs:1331
5072024-11-28T01:33:23.286ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5082024-11-28T01:33:23.286ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c7ca3a79-57f5-4e47-8807-03bd67966814/crypt/zone/oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3"), full_name: "oxp_c7ca3a79-57f5-4e47-8807-03bd67966814/crypt/zone/oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3" } file = sled-storage/src/manager.rs:1331
5092024-11-28T01:33:23.406ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5102024-11-28T01:33:23.406ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_977ab5dc-c743-4525-b172-ecf8a9034abe/crucible" } file = sled-storage/src/manager.rs:1331
5112024-11-28T01:33:23.517ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5122024-11-28T01:33:23.517ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_0176efd8-6a11-46a9-8bcd-a97e447fcb50/crypt/internal_dns" } file = sled-storage/src/manager.rs:1331
5132024-11-28T01:33:23.590ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/12368a57-a643-445a-b32d-da10700bd20e) file = sled-agent/src/probe_manager.rs:231 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
5142024-11-28T01:33:23.629ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5152024-11-28T01:33:23.629ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/75598bf6-d28b-4334-94db-398fc2a8b539/crypt/debug"), full_name: "oxp_75598bf6-d28b-4334-94db-398fc2a8b539/crypt/debug" } file = sled-storage/src/manager.rs:1331
5162024-11-28T01:33:23.713ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5172024-11-28T01:33:23.713ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8164ba35-0eff-436a-9b6b-80912419e510/crypt/zone/oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2"), full_name: "oxp_8164ba35-0eff-436a-9b6b-80912419e510/crypt/zone/oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2" } file = sled-storage/src/manager.rs:1331
5182024-11-28T01:33:23.836ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5192024-11-28T01:33:23.836ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/internal_dns" } file = sled-storage/src/manager.rs:1331
5202024-11-28T01:33:23.945ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5212024-11-28T01:33:23.945ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_8164ba35-0eff-436a-9b6b-80912419e510/crypt/internal_dns" } file = sled-storage/src/manager.rs:1331
5222024-11-28T01:33:24.053ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5232024-11-28T01:33:24.053ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_0176efd8-6a11-46a9-8bcd-a97e447fcb50/crucible" } file = sled-storage/src/manager.rs:1331
5242024-11-28T01:33:24.158ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5252024-11-28T01:33:24.158ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_2fcc59ee-335a-4c80-bbcf-e66263237ec3/crucible" } file = sled-storage/src/manager.rs:1331
5262024-11-28T01:33:24.266ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5272024-11-28T01:33:24.266ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/zone/oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038"), full_name: "oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/zone/oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038" } file = sled-storage/src/manager.rs:1331
5282024-11-28T01:33:24.381ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5292024-11-28T01:33:24.381ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/216077bb-f8f0-4fbf-8aa0-1db2249e65da/crypt/zone/oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e"), full_name: "oxp_216077bb-f8f0-4fbf-8aa0-1db2249e65da/crypt/zone/oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e" } file = sled-storage/src/manager.rs:1331
5302024-11-28T01:33:24.485ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 20.337120246s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
5312024-11-28T01:33:24.499ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5322024-11-28T01:33:24.499ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_216077bb-f8f0-4fbf-8aa0-1db2249e65da/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
5332024-11-28T01:33:24.591ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/12368a57-a643-445a-b32d-da10700bd20e) file = sled-agent/src/probe_manager.rs:231 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
5342024-11-28T01:33:24.609ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5352024-11-28T01:33:24.609ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/216077bb-f8f0-4fbf-8aa0-1db2249e65da/crypt/debug"), full_name: "oxp_216077bb-f8f0-4fbf-8aa0-1db2249e65da/crypt/debug" } file = sled-storage/src/manager.rs:1331
5362024-11-28T01:33:24.692ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5372024-11-28T01:33:24.692ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/216077bb-f8f0-4fbf-8aa0-1db2249e65da/crypt/zone"), full_name: "oxp_216077bb-f8f0-4fbf-8aa0-1db2249e65da/crypt/zone" } file = sled-storage/src/manager.rs:1331
5382024-11-28T01:33:24.796ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5392024-11-28T01:33:24.796ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/zone/oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26"), full_name: "oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/zone/oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26" } file = sled-storage/src/manager.rs:1331
5402024-11-28T01:33:24.916ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5412024-11-28T01:33:24.916ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_8164ba35-0eff-436a-9b6b-80912419e510/crucible" } file = sled-storage/src/manager.rs:1331
5422024-11-28T01:33:25.027ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5432024-11-28T01:33:25.027ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/clickhouse" } file = sled-storage/src/manager.rs:1331
5442024-11-28T01:33:25.136ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5452024-11-28T01:33:25.136ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_beaaa347-4478-48b4-b888-a5873241ded0/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
5462024-11-28T01:33:25.249ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5472024-11-28T01:33:25.249ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/zone/oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd"), full_name: "oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/zone/oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd" } file = sled-storage/src/manager.rs:1331
5482024-11-28T01:33:25.369ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5492024-11-28T01:33:25.369ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
5502024-11-28T01:33:25.481ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5512024-11-28T01:33:25.481ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/external_dns" } file = sled-storage/src/manager.rs:1331
5522024-11-28T01:33:25.592ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5532024-11-28T01:33:25.592ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_8164ba35-0eff-436a-9b6b-80912419e510/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
5542024-11-28T01:33:25.596ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/12368a57-a643-445a-b32d-da10700bd20e) file = sled-agent/src/probe_manager.rs:231 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
5552024-11-28T01:33:25.706ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5562024-11-28T01:33:25.706ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/zone/oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048"), full_name: "oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/zone/oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048" } file = sled-storage/src/manager.rs:1331
5572024-11-28T01:33:25.829ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5582024-11-28T01:33:25.829ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee/crucible" } file = sled-storage/src/manager.rs:1331
5592024-11-28T01:33:25.934ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5602024-11-28T01:33:25.934ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8164ba35-0eff-436a-9b6b-80912419e510/crypt/zone/oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143"), full_name: "oxp_8164ba35-0eff-436a-9b6b-80912419e510/crypt/zone/oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143" } file = sled-storage/src/manager.rs:1331
5612024-11-28T01:33:26.052ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5622024-11-28T01:33:26.052ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/2fcc59ee-335a-4c80-bbcf-e66263237ec3/crypt/zone/oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389"), full_name: "oxp_2fcc59ee-335a-4c80-bbcf-e66263237ec3/crypt/zone/oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389" } file = sled-storage/src/manager.rs:1331
5632024-11-28T01:33:26.168ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5642024-11-28T01:33:26.168ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8164ba35-0eff-436a-9b6b-80912419e510/crypt/zone/oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9"), full_name: "oxp_8164ba35-0eff-436a-9b6b-80912419e510/crypt/zone/oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9" } file = sled-storage/src/manager.rs:1331
5652024-11-28T01:33:26.229ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists - OK file = sled-agent/src/services.rs:4292
5662024-11-28T01:33:26.229ZINFOSledAgent (ServiceManager): Ensuring there is a default route file = sled-agent/src/services.rs:4302 gateway = fd00:1122:3344:101::1
5672024-11-28T01:33:26.237ZINFOSledAgent (ServiceManager): Default route already exists file = sled-agent/src/services.rs:4316 gateway = fd00:1122:3344:101::1
5682024-11-28T01:33:26.237ZINFOSledAgent (ServiceManager): configuring dendrite service file = sled-agent/src/services.rs:4381
5692024-11-28T01:33:26.298ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5702024-11-28T01:33:26.298ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8164ba35-0eff-436a-9b6b-80912419e510/crypt/debug"), full_name: "oxp_8164ba35-0eff-436a-9b6b-80912419e510/crypt/debug" } file = sled-storage/src/manager.rs:1331
5712024-11-28T01:33:26.387ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5722024-11-28T01:33:26.387ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/977ab5dc-c743-4525-b172-ecf8a9034abe/crypt/debug"), full_name: "oxp_977ab5dc-c743-4525-b172-ecf8a9034abe/crypt/debug" } file = sled-storage/src/manager.rs:1331
5732024-11-28T01:33:26.394ZINFOSledAgent (ServiceManager): refreshed dendrite service with new configuration file = sled-agent/src/services.rs:4447
5742024-11-28T01:33:26.394ZINFOSledAgent (ServiceManager): configuring lldp service file = sled-agent/src/services.rs:4474
5752024-11-28T01:33:26.436ZINFOSledAgent (ServiceManager): refreshed lldpd service with new configuration file = sled-agent/src/services.rs:4487
5762024-11-28T01:33:26.437ZINFOSledAgent (ServiceManager): configuring MGS service file = sled-agent/src/services.rs:4333
5772024-11-28T01:33:26.475ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5782024-11-28T01:33:26.475ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/debug"), full_name: "oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/debug" } file = sled-storage/src/manager.rs:1331
5792024-11-28T01:33:26.490ZINFOSledAgent (ServiceManager): refreshed MGS service with new configuration file = sled-agent/src/services.rs:4375
5802024-11-28T01:33:26.490ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:4454 rack_subnet = fd00:1122:3344::
5812024-11-28T01:33:26.513ZINFOSledAgent (ServiceManager): refreshed wicketd service with new configuration file = sled-agent/src/services.rs:4465
5822024-11-28T01:33:26.513ZINFOSledAgent (ServiceManager): configuring mgd service file = sled-agent/src/services.rs:4507
5832024-11-28T01:33:26.517ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9783695383746708995 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
5842024-11-28T01:33:26.517ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 3449068533422146992 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
5852024-11-28T01:33:26.565ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5862024-11-28T01:33:26.565ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c7ca3a79-57f5-4e47-8807-03bd67966814/crypt/zone"), full_name: "oxp_c7ca3a79-57f5-4e47-8807-03bd67966814/crypt/zone" } file = sled-storage/src/manager.rs:1331
5872024-11-28T01:33:26.586ZINFOSledAgent (ServiceManager): refreshed mgd service with new configuration file = sled-agent/src/services.rs:4539
5882024-11-28T01:33:26.586ZINFOSledAgent (ServiceManager): configuring mg-ddm service file = sled-agent/src/services.rs:4545
5892024-11-28T01:33:26.601ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/12368a57-a643-445a-b32d-da10700bd20e) file = sled-agent/src/probe_manager.rs:231 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
5902024-11-28T01:33:26.657ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5912024-11-28T01:33:26.657ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0176efd8-6a11-46a9-8bcd-a97e447fcb50/crypt/zone/oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f"), full_name: "oxp_0176efd8-6a11-46a9-8bcd-a97e447fcb50/crypt/zone/oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f" } file = sled-storage/src/manager.rs:1331
5922024-11-28T01:33:26.683ZINFOSledAgent (ServiceManager): refreshed mg-ddm service with new configuration file = sled-agent/src/services.rs:4586
5932024-11-28T01:33:26.683ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:370
5942024-11-28T01:33:26.728ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:420
5952024-11-28T01:33:26.728ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/12368a57-a643-445a-b32d-da10700bd20e) file = sled-agent/src/nexus.rs:527 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
5962024-11-28T01:33:26.769ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:717
5972024-11-28T01:33:26.769ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:762
5982024-11-28T01:33:26.770ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:765 uptime = 1
5992024-11-28T01:33:26.770ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.1.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: Some(\n None,\n ),\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n tx_eq: None,\n },\n },\n },\n} file = sled-agent/src/bootstrap/early_networking.rs:444
6002024-11-28T01:33:26.782ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6012024-11-28T01:33:26.782ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/2fcc59ee-335a-4c80-bbcf-e66263237ec3/crypt/zone/oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80"), full_name: "oxp_2fcc59ee-335a-4c80-bbcf-e66263237ec3/crypt/zone/oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80" } file = sled-storage/src/manager.rs:1331
6022024-11-28T01:33:26.815ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
6032024-11-28T01:33:26.906ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6042024-11-28T01:33:26.906ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0176efd8-6a11-46a9-8bcd-a97e447fcb50/crypt/zone"), full_name: "oxp_0176efd8-6a11-46a9-8bcd-a97e447fcb50/crypt/zone" } file = sled-storage/src/manager.rs:1331
6052024-11-28T01:33:26.989ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6062024-11-28T01:33:26.989ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/zone"), full_name: "oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/zone" } file = sled-storage/src/manager.rs:1331
6072024-11-28T01:33:27.072ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6082024-11-28T01:33:27.072ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/2fcc59ee-335a-4c80-bbcf-e66263237ec3/crypt/zone"), full_name: "oxp_2fcc59ee-335a-4c80-bbcf-e66263237ec3/crypt/zone" } file = sled-storage/src/manager.rs:1331
6092024-11-28T01:33:27.154ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6102024-11-28T01:33:27.154ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_beaaa347-4478-48b4-b888-a5873241ded0/crucible" } file = sled-storage/src/manager.rs:1331
6112024-11-28T01:33:27.262ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6122024-11-28T01:33:27.262ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/216077bb-f8f0-4fbf-8aa0-1db2249e65da/crypt/zone/oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1"), full_name: "oxp_216077bb-f8f0-4fbf-8aa0-1db2249e65da/crypt/zone/oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1" } file = sled-storage/src/manager.rs:1331
6132024-11-28T01:33:27.387ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6142024-11-28T01:33:27.388ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8164ba35-0eff-436a-9b6b-80912419e510/crypt/zone"), full_name: "oxp_8164ba35-0eff-436a-9b6b-80912419e510/crypt/zone" } file = sled-storage/src/manager.rs:1331
6152024-11-28T01:33:27.443ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 6865697397444298559 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
6162024-11-28T01:33:27.443ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17855345072000113602 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
6172024-11-28T01:33:27.470ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6182024-11-28T01:33:27.470ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0176efd8-6a11-46a9-8bcd-a97e447fcb50/crypt/debug"), full_name: "oxp_0176efd8-6a11-46a9-8bcd-a97e447fcb50/crypt/debug" } file = sled-storage/src/manager.rs:1331
6192024-11-28T01:33:27.554ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6202024-11-28T01:33:27.554ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/beaaa347-4478-48b4-b888-a5873241ded0/crypt/zone/oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4"), full_name: "oxp_beaaa347-4478-48b4-b888-a5873241ded0/crypt/zone/oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4" } file = sled-storage/src/manager.rs:1331
6212024-11-28T01:33:27.604ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/12368a57-a643-445a-b32d-da10700bd20e) file = sled-agent/src/probe_manager.rs:231 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
6222024-11-28T01:33:27.678ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6232024-11-28T01:33:27.678ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0176efd8-6a11-46a9-8bcd-a97e447fcb50/crypt/zone/oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014"), full_name: "oxp_0176efd8-6a11-46a9-8bcd-a97e447fcb50/crypt/zone/oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014" } file = sled-storage/src/manager.rs:1331
6242024-11-28T01:33:27.802ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6252024-11-28T01:33:27.802ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_75598bf6-d28b-4334-94db-398fc2a8b539/crucible" } file = sled-storage/src/manager.rs:1331
6262024-11-28T01:33:27.913ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6272024-11-28T01:33:27.913ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/zone/oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246"), full_name: "oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/zone/oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246" } file = sled-storage/src/manager.rs:1331
6282024-11-28T01:33:28.042ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6292024-11-28T01:33:28.042ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c7ca3a79-57f5-4e47-8807-03bd67966814/crypt/zone/oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917"), full_name: "oxp_c7ca3a79-57f5-4e47-8807-03bd67966814/crypt/zone/oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917" } file = sled-storage/src/manager.rs:1331
6302024-11-28T01:33:28.167ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6312024-11-28T01:33:28.167ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c7ca3a79-57f5-4e47-8807-03bd67966814/crypt/debug"), full_name: "oxp_c7ca3a79-57f5-4e47-8807-03bd67966814/crypt/debug" } file = sled-storage/src/manager.rs:1331
6322024-11-28T01:33:28.252ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/.omicron-datasets.json.tmp file = common/src/ledger.rs:199 request = datasets_ensure
6332024-11-28T01:33:28.253ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/.omicron-datasets.json.tmp file = common/src/ledger.rs:199 request = datasets_ensure
6342024-11-28T01:33:28.253ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:603
6352024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
6362024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
6372024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
6382024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
6392024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
6402024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
6412024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
6422024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
6432024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
6442024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
6452024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
6462024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
6472024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
6482024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
6492024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
6502024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
6512024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
6522024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
6532024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
6542024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
6552024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
6562024-11-28T01:33:28.253ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
6572024-11-28T01:33:28.253ZINFOSledAgent: datasets ensure: Updated storage file = sled-agent/src/sled_agent.rs:864 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
6582024-11-28T01:33:28.253ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7159278 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:61855 req_id = 089342b7-4083-4d6e-ac7e-de5b11358cd3 response_code = 200 uri = /datasets
6592024-11-28T01:33:28.285ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: 98733a1a-8c66-4d41-88fc-32826e30dc26 (service), filesystem_pool: Some(ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: e7a3f401-14e4-47b9-8060-4cbbfc840143 (service), filesystem_pool: Some(ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: f9751d6f-77c7-44d3-9105-87b0872eec3f (service), filesystem_pool: Some(ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:525 sled_address = [fd00:1122:3344:101::1]:12345
6602024-11-28T01:33:28.285ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:46345
6612024-11-28T01:33:28.286ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 98733a1a-8c66-4d41-88fc-32826e30dc26, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
6622024-11-28T01:33:28.286ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 98733a1a-8c66-4d41-88fc-32826e30dc26, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: InternalDns }, responder: ... }
6632024-11-28T01:33:28.321ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e7a3f401-14e4-47b9-8060-4cbbfc840143, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
6642024-11-28T01:33:28.321ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: e7a3f401-14e4-47b9-8060-4cbbfc840143, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6652024-11-28T01:33:28.355ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f9751d6f-77c7-44d3-9105-87b0872eec3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
6662024-11-28T01:33:28.355ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: f9751d6f-77c7-44d3-9105-87b0872eec3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6672024-11-28T01:33:28.391ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6682024-11-28T01:33:28.391ZINFOSledAgent (ServiceManager): No ledger in /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/omicron-zones.json file = common/src/ledger.rs:188
6692024-11-28T01:33:28.391ZINFOSledAgent (ServiceManager): No ledger in /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/omicron-zones.json file = common/src/ledger.rs:188
6702024-11-28T01:33:28.391ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6712024-11-28T01:33:28.437ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6722024-11-28T01:33:28.437ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6732024-11-28T01:33:28.437ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6742024-11-28T01:33:28.486ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143 file = illumos-utils/src/zone.rs:325
6752024-11-28T01:33:28.490ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26 file = illumos-utils/src/zone.rs:325
6762024-11-28T01:33:28.494ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f file = illumos-utils/src/zone.rs:325
6772024-11-28T01:33:28.507ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143 file = illumos-utils/src/zone.rs:363
6782024-11-28T01:33:28.512ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26 file = illumos-utils/src/zone.rs:363
6792024-11-28T01:33:28.516ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f file = illumos-utils/src/zone.rs:363
6802024-11-28T01:33:28.605ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/12368a57-a643-445a-b32d-da10700bd20e) file = sled-agent/src/probe_manager.rs:231 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
6812024-11-28T01:33:28.729ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/12368a57-a643-445a-b32d-da10700bd20e) file = sled-agent/src/nexus.rs:527 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
6822024-11-28T01:33:29.607ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/12368a57-a643-445a-b32d-da10700bd20e) file = sled-agent/src/probe_manager.rs:231 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
6832024-11-28T01:33:30.092ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService1'/> <propval type="astring" name="gateway" value='fd00:1122:3344:2::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:2::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:2::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:2::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6842024-11-28T01:33:30.092ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:2::, width: 64 }
6852024-11-28T01:33:30.092ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143
6862024-11-28T01:33:30.580ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService2'/> <propval type="astring" name="gateway" value='fd00:1122:3344:1::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:1::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:1::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:1::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6872024-11-28T01:33:31.061ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:1::, width: 64 }
6882024-11-28T01:33:31.061ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService3'/> <propval type="astring" name="gateway" value='fd00:1122:3344:3::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:3::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:3::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:3::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6892024-11-28T01:33:31.061ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:3::, width: 64 }
6902024-11-28T01:33:31.070ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143") failed: Property not found. retry in 29.643126ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143
6912024-11-28T01:33:31.070ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26
6922024-11-28T01:33:31.075ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f
6932024-11-28T01:33:31.110ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143") failed: Property not found. retry in 75.598201ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143
6942024-11-28T01:33:31.196ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143") failed: Property not found. retry in 222.133013ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143
6952024-11-28T01:33:31.205ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f") failed: Property not found. retry in 42.157217ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f
6962024-11-28T01:33:31.212ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26") failed: Property not found. retry in 65.913886ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26
6972024-11-28T01:33:31.255ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f") failed: Property not found. retry in 126.330173ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f
6982024-11-28T01:33:31.374ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26") failed: Property not found. retry in 100.744035ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26
6992024-11-28T01:33:31.429ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f") failed: Property not found. retry in 147.876062ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f
7002024-11-28T01:33:31.498ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143") failed: Property not found. retry in 580.058425ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143
7012024-11-28T01:33:31.567ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26") failed: Property not found. retry in 205.493995ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26
7022024-11-28T01:33:31.611ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f") failed: Property not found. retry in 383.610684ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f
7032024-11-28T01:33:31.782ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26") failed: Property not found. retry in 261.564552ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26
7042024-11-28T01:33:32.004ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f") failed: Property not found. retry in 715.810884ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f
7052024-11-28T01:33:32.054ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26") failed: Property not found. retry in 1.193384378s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26
7062024-11-28T01:33:32.089ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143") failed: Property not found. retry in 701.160801ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143
7072024-11-28T01:33:32.730ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f") failed: Property not found. retry in 1.239485019s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f
7082024-11-28T01:33:32.799ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143") failed: Property not found. retry in 522.1682ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143
7092024-11-28T01:33:33.257ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26") failed: Property not found. retry in 539.268305ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26
7102024-11-28T01:33:33.331ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143") failed: Property not found. retry in 761.000058ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143
7112024-11-28T01:33:33.806ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26") failed: Property not found. retry in 874.112771ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26
7122024-11-28T01:33:33.978ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f") failed: Property not found. retry in 1.081184312s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f
7132024-11-28T01:33:34.101ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143") failed: Property not found. retry in 872.509839ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143
7142024-11-28T01:33:34.689ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26") failed: Property not found. retry in 556.079186ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26
7152024-11-28T01:33:34.983ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143") failed: Property not found. retry in 1.251051208s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143
7162024-11-28T01:33:35.069ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f") failed: Property not found. retry in 678.553272ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f
7172024-11-28T01:33:35.254ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26") failed: Property not found. retry in 1.431937647s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26
7182024-11-28T01:33:35.756ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f") failed: Property not found. retry in 1.293408691s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f
7192024-11-28T01:33:36.520ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9783695383746708995 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
7202024-11-28T01:33:36.520ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 3449068533422146992 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
7212024-11-28T01:33:37.070ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_internal_dns_e7a3f401-14e4-47b9-8060-4cbbfc840143
7222024-11-28T01:33:37.070ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_internal_dns_98733a1a-8c66-4d41-88fc-32826e30dc26
7232024-11-28T01:33:37.070ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_internal_dns_f9751d6f-77c7-44d3-9105-87b0872eec3f
7242024-11-28T01:33:37.070ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7252024-11-28T01:33:37.071ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7262024-11-28T01:33:37.071ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8785436 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:46345 req_id = a0757557-d27d-4365-b2c9-06928f4ed5bb response_code = 204 uri = /omicron-zones
7272024-11-28T01:33:37.073ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:642
7282024-11-28T01:33:37.380ZWARNSledAgent (RSS): failed to write DNS configuration (will retry in 89.64092ms) dns_config_addr = [fd00:1122:3344:1::1]:5353 error_message = Communication Error: error sending request for url (http://[fd00:1122:3344:1::1]:5353/config) file = sled-agent/src/rack_setup/service.rs:662
7292024-11-28T01:33:37.444ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 6865697397444298559 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
7302024-11-28T01:33:37.445ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17855345072000113602 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
7312024-11-28T01:33:37.495ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:642
7322024-11-28T01:33:37.555ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:642
7332024-11-28T01:33:37.610ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:678
7342024-11-28T01:33:37.611ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
7352024-11-28T01:33:37.641ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
7362024-11-28T01:33:37.642ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
7372024-11-28T01:33:37.642ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
7382024-11-28T01:33:37.671ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: 98733a1a-8c66-4d41-88fc-32826e30dc26 (service), filesystem_pool: Some(ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: e7a3f401-14e4-47b9-8060-4cbbfc840143 (service), filesystem_pool: Some(ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: f9751d6f-77c7-44d3-9105-87b0872eec3f (service), filesystem_pool: Some(ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 8ff676ac-4433-41a4-8ead-2afb6c570229 (service), filesystem_pool: Some(ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: dc5c6f99-a9da-4946-a74a-4accf50cbd91, kind: Service { id: 8ff676ac-4433-41a4-8ead-2afb6c570229 }, name: Name("ntp-8ff676ac-4433-41a4-8ead-2afb6c570229"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 184])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:525 sled_address = [fd00:1122:3344:101::1]:12345
7392024-11-28T01:33:37.671ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:52486
7402024-11-28T01:33:37.671ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 98733a1a-8c66-4d41-88fc-32826e30dc26, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
7412024-11-28T01:33:37.671ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 98733a1a-8c66-4d41-88fc-32826e30dc26, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: InternalDns }, responder: ... }
7422024-11-28T01:33:37.709ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e7a3f401-14e4-47b9-8060-4cbbfc840143, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
7432024-11-28T01:33:37.709ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: e7a3f401-14e4-47b9-8060-4cbbfc840143, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7442024-11-28T01:33:37.747ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f9751d6f-77c7-44d3-9105-87b0872eec3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
7452024-11-28T01:33:37.747ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: f9751d6f-77c7-44d3-9105-87b0872eec3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7462024-11-28T01:33:37.783ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
7472024-11-28T01:33:37.783ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/omicron-zones.json file = common/src/ledger.rs:177
7482024-11-28T01:33:37.783ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/omicron-zones.json file = common/src/ledger.rs:177
7492024-11-28T01:33:37.784ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
7502024-11-28T01:33:37.795ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
7512024-11-28T01:33:37.829ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
7522024-11-28T01:33:37.829ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
7532024-11-28T01:33:37.829ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
7542024-11-28T01:33:37.859ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte0", ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 168, 184]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 } }) }
7552024-11-28T01:33:37.859ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1263 zone_type = boundary_ntp
7562024-11-28T01:33:37.860ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.169 file = clients/dpd-client/src/lib.rs:83
7572024-11-28T01:33:37.860ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.169 file = clients/dpd-client/src/lib.rs:90
7582024-11-28T01:33:37.860ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.169 file = clients/dpd-client/src/lib.rs:118
7592024-11-28T01:33:37.860ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
7602024-11-28T01:33:37.885ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229 file = illumos-utils/src/zone.rs:325
7612024-11-28T01:33:37.904ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229 file = illumos-utils/src/zone.rs:363
7622024-11-28T01:33:38.519ZINFOSledAgent (ServiceManager): Profile for oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService4'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/chrony-setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="allow" value='fd00:1122:3344:100::/56'/> <propval type="boolean" name="boundary" value='true'/> <propval type="astring" name="boundary_pool" value='boundary-ntp.control-plane.oxide.internal'/> <propval type="astring" name="server" value='0.pool.ntp.org'/> </property_group> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="1.1.1.1" /> <value_node value="9.9.9.9" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/ntp"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte0'/> <propval type="astring" name="gateway" value='172.30.3.1'/> <propval type="astring" name="ip" value='172.30.3.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7632024-11-28T01:33:38.519ZINFOSledAgent (ServiceManager): Booting oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229
7642024-11-28T01:33:38.652ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229") failed: Property not found. retry in 57.352337ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229
7652024-11-28T01:33:38.718ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229") failed: Property not found. retry in 108.382539ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229
7662024-11-28T01:33:38.838ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229") failed: Property not found. retry in 229.680882ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229
7672024-11-28T01:33:39.081ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229") failed: Property not found. retry in 501.947565ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229
7682024-11-28T01:33:39.593ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229") failed: Property not found. retry in 573.244824ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229
7692024-11-28T01:33:40.177ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229") failed: Property not found. retry in 1.122364322s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229
7702024-11-28T01:33:41.309ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229") failed: Property not found. retry in 598.322969ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229
7712024-11-28T01:33:41.917ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229") failed: Property not found. retry in 1.038082664s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229
7722024-11-28T01:33:42.965ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229") failed: Property not found. retry in 1.091326305s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229
7732024-11-28T01:33:44.077ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229
7742024-11-28T01:33:44.077ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7752024-11-28T01:33:44.077ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7762024-11-28T01:33:44.078ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6406411 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:52486 req_id = 24f12ffc-4ff1-459e-a909-4c934cb75cf8 response_code = 204 uri = /omicron-zones
7772024-11-28T01:33:44.080ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:719
7782024-11-28T01:33:44.111ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:699
7792024-11-28T01:33:44.111ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:34631
7802024-11-28T01:33:44.120ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3900
7812024-11-28T01:33:44.120ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 8740 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:34631 req_id = 958ac870-892e-473f-8f48-7a11a70ce458 response_code = 500 uri = /timesync
7822024-11-28T01:33:44.122ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:749
7832024-11-28T01:33:44.216ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:699
7842024-11-28T01:33:44.216ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:54307
7852024-11-28T01:33:46.260ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 11971999552025750697 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
7862024-11-28T01:33:46.520ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9783695383746708995 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
7872024-11-28T01:33:46.521ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 3449068533422146992 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
7882024-11-28T01:33:46.710ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 55646998708649281 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
7892024-11-28T01:33:47.073ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 7148868828734122020 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
7902024-11-28T01:33:47.446ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 6865697397444298559 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
7912024-11-28T01:33:47.446ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17855345072000113602 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
7922024-11-28T01:33:48.225ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3900
7932024-11-28T01:33:48.226ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 4009178 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:54307 req_id = 7be7abbe-527a-4ff1-9feb-0fa08bffdede response_code = 500 uri = /timesync
7942024-11-28T01:33:48.228ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:749
7952024-11-28T01:33:48.390ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:699
7962024-11-28T01:33:48.390ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57109
7972024-11-28T01:33:48.398ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_8ff676ac-4433-41a4-8ead-2afb6c570229': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3900
7982024-11-28T01:33:48.399ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 8782 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:57109 req_id = 35a0fa60-e6cd-4b90-8ede-04cb452549fd response_code = 500 uri = /timesync
7992024-11-28T01:33:48.402ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:749
8002024-11-28T01:33:48.614ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:699
8012024-11-28T01:33:48.614ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64987
8022024-11-28T01:33:48.623ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8316 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:64987 req_id = 4e4f34a9-c9c7-4c05-b1af-0a70824b47ac response_code = 200 uri = /timesync
8032024-11-28T01:33:48.625ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:727
8042024-11-28T01:33:48.625ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:749
8052024-11-28T01:33:48.774ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:699
8062024-11-28T01:33:48.775ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:45952
8072024-11-28T01:33:48.784ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9258 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:45952 req_id = d218ee90-fbb7-4c65-bdce-b89f0ba2a9b4 response_code = 200 uri = /timesync
8082024-11-28T01:33:48.786ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:727
8092024-11-28T01:33:48.786ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:749
8102024-11-28T01:33:49.055ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:699
8112024-11-28T01:33:49.056ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:43040
8122024-11-28T01:33:49.064ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8775 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:43040 req_id = 30d0bd19-673d-4ab7-be37-c0ae55f778bb response_code = 200 uri = /timesync
8132024-11-28T01:33:49.067ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:727
8142024-11-28T01:33:49.067ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:749
8152024-11-28T01:33:49.379ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:699
8162024-11-28T01:33:49.380ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:44177
8172024-11-28T01:33:49.388ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8416 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:44177 req_id = 022c5b59-31d5-4c59-b011-78db61536b54 response_code = 200 uri = /timesync
8182024-11-28T01:33:49.391ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:727
8192024-11-28T01:33:49.391ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:749
8202024-11-28T01:33:49.705ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:699
8212024-11-28T01:33:49.705ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:34625
8222024-11-28T01:33:49.714ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8155 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:34625 req_id = 430aa7cf-781e-4d48-b54e-53180c6f7d55 response_code = 200 uri = /timesync
8232024-11-28T01:33:49.717ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:727
8242024-11-28T01:33:49.717ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:749
8252024-11-28T01:33:50.281ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:699
8262024-11-28T01:33:50.281ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:41858
8272024-11-28T01:33:50.289ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8310 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:41858 req_id = ee3a19bd-99ea-4bb3-9a35-3b887f058626 response_code = 200 uri = /timesync
8282024-11-28T01:33:50.294ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:727
8292024-11-28T01:33:50.294ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:749
8302024-11-28T01:33:50.213ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:699
8312024-11-28T01:33:50.213ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:44785
8322024-11-28T01:33:50.235ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 21871 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:44785 req_id = 8ccd6655-351f-4f58-8585-cd4dbf4939f7 response_code = 200 uri = /timesync
8332024-11-28T01:33:50.240ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 2901090547, ip_addr: 172.235.32.243, stratum: 3, ref_time: 1732757629.9460597, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:727
8342024-11-28T01:33:50.240ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1412
8352024-11-28T01:33:50.270ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 98733a1a-8c66-4d41-88fc-32826e30dc26 (service), filesystem_pool: Some(ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: e7a3f401-14e4-47b9-8060-4cbbfc840143 (service), filesystem_pool: Some(ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: f9751d6f-77c7-44d3-9105-87b0872eec3f (service), filesystem_pool: Some(ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 8ff676ac-4433-41a4-8ead-2afb6c570229 (service), filesystem_pool: Some(ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: dc5c6f99-a9da-4946-a74a-4accf50cbd91, kind: Service { id: 8ff676ac-4433-41a4-8ead-2afb6c570229 }, name: Name("ntp-8ff676ac-4433-41a4-8ead-2afb6c570229"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 184])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 057f14d6-21ef-4a1a-9443-743301729246 (service), filesystem_pool: Some(ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External } } } }, OmicronZoneConfig { id: 789268ca-0469-4ab5-abce-6379b90792b9 (service), filesystem_pool: Some(ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External } } } }, OmicronZoneConfig { id: 0df55fc2-6b39-4a29-9df0-909dac438014 (service), filesystem_pool: Some(ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External } } } }, OmicronZoneConfig { id: 99301c48-601f-449e-8e73-447a87265d28 (service), filesystem_pool: Some(ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 777baf9d-7c4a-4b11-b86c-ebb4af23f5f1 (service), filesystem_pool: Some(ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:525 sled_address = [fd00:1122:3344:101::1]:12345
8362024-11-28T01:33:50.270ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:37535
8372024-11-28T01:33:50.270ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 98733a1a-8c66-4d41-88fc-32826e30dc26, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8382024-11-28T01:33:50.270ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 98733a1a-8c66-4d41-88fc-32826e30dc26, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: InternalDns }, responder: ... }
8392024-11-28T01:33:50.308ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e7a3f401-14e4-47b9-8060-4cbbfc840143, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8402024-11-28T01:33:50.308ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: e7a3f401-14e4-47b9-8060-4cbbfc840143, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8412024-11-28T01:33:50.344ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f9751d6f-77c7-44d3-9105-87b0872eec3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8422024-11-28T01:33:50.344ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: f9751d6f-77c7-44d3-9105-87b0872eec3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8432024-11-28T01:33:50.379ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 057f14d6-21ef-4a1a-9443-743301729246, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8442024-11-28T01:33:50.379ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 057f14d6-21ef-4a1a-9443-743301729246, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Cockroach }, responder: ... }
8452024-11-28T01:33:50.416ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 789268ca-0469-4ab5-abce-6379b90792b9, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8462024-11-28T01:33:50.416ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 789268ca-0469-4ab5-abce-6379b90792b9, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8472024-11-28T01:33:50.454ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0df55fc2-6b39-4a29-9df0-909dac438014, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8482024-11-28T01:33:50.454ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 0df55fc2-6b39-4a29-9df0-909dac438014, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8492024-11-28T01:33:50.490ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 99301c48-601f-449e-8e73-447a87265d28, dataset_name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8502024-11-28T01:33:50.490ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 99301c48-601f-449e-8e73-447a87265d28, dataset_name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8512024-11-28T01:33:50.527ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 777baf9d-7c4a-4b11-b86c-ebb4af23f5f1, dataset_name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8522024-11-28T01:33:50.527ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 777baf9d-7c4a-4b11-b86c-ebb4af23f5f1, dataset_name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: Cockroach }, responder: ... }
8532024-11-28T01:33:50.562ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8542024-11-28T01:33:50.562ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/omicron-zones.json file = common/src/ledger.rs:177
8552024-11-28T01:33:50.562ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/omicron-zones.json file = common/src/ledger.rs:177
8562024-11-28T01:33:50.562ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8572024-11-28T01:33:50.595ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28
8582024-11-28T01:33:50.595ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014
8592024-11-28T01:33:50.595ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1
8602024-11-28T01:33:50.595ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246
8612024-11-28T01:33:50.595ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9
8622024-11-28T01:33:50.595ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28", TimeNotSynchronized), ("oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014", TimeNotSynchronized), ("oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1", TimeNotSynchronized), ("oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246", TimeNotSynchronized), ("oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 324778 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:37535 req_id = 9ae00659-ff18-46bf-8a36-9dd1b3922c06 response_code = 500 uri = /omicron-zones
8632024-11-28T01:33:50.595ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "9ae00659-ff18-46bf-8a36-9dd1b3922c06", "content-length": "124", "date": "Thu, 28 Nov 2024 01:33:49 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "9ae00659-ff18-46bf-8a36-9dd1b3922c06" } file = sled-agent/src/rack_setup/service.rs:562 retry_after = 142.560567ms sled_address = [fd00:1122:3344:101::1]:12345
8642024-11-28T01:33:50.738ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 98733a1a-8c66-4d41-88fc-32826e30dc26 (service), filesystem_pool: Some(ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: e7a3f401-14e4-47b9-8060-4cbbfc840143 (service), filesystem_pool: Some(ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: f9751d6f-77c7-44d3-9105-87b0872eec3f (service), filesystem_pool: Some(ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 8ff676ac-4433-41a4-8ead-2afb6c570229 (service), filesystem_pool: Some(ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: dc5c6f99-a9da-4946-a74a-4accf50cbd91, kind: Service { id: 8ff676ac-4433-41a4-8ead-2afb6c570229 }, name: Name("ntp-8ff676ac-4433-41a4-8ead-2afb6c570229"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 184])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 057f14d6-21ef-4a1a-9443-743301729246 (service), filesystem_pool: Some(ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External } } } }, OmicronZoneConfig { id: 789268ca-0469-4ab5-abce-6379b90792b9 (service), filesystem_pool: Some(ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External } } } }, OmicronZoneConfig { id: 0df55fc2-6b39-4a29-9df0-909dac438014 (service), filesystem_pool: Some(ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External } } } }, OmicronZoneConfig { id: 99301c48-601f-449e-8e73-447a87265d28 (service), filesystem_pool: Some(ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 777baf9d-7c4a-4b11-b86c-ebb4af23f5f1 (service), filesystem_pool: Some(ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:525 sled_address = [fd00:1122:3344:101::1]:12345
8652024-11-28T01:33:50.739ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 98733a1a-8c66-4d41-88fc-32826e30dc26, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8662024-11-28T01:33:50.739ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 98733a1a-8c66-4d41-88fc-32826e30dc26, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: InternalDns }, responder: ... }
8672024-11-28T01:33:50.776ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e7a3f401-14e4-47b9-8060-4cbbfc840143, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8682024-11-28T01:33:50.776ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: e7a3f401-14e4-47b9-8060-4cbbfc840143, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8692024-11-28T01:33:50.811ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f9751d6f-77c7-44d3-9105-87b0872eec3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8702024-11-28T01:33:50.811ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: f9751d6f-77c7-44d3-9105-87b0872eec3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8712024-11-28T01:33:50.846ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 057f14d6-21ef-4a1a-9443-743301729246, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8722024-11-28T01:33:50.846ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 057f14d6-21ef-4a1a-9443-743301729246, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Cockroach }, responder: ... }
8732024-11-28T01:33:50.881ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 789268ca-0469-4ab5-abce-6379b90792b9, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8742024-11-28T01:33:50.881ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 789268ca-0469-4ab5-abce-6379b90792b9, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8752024-11-28T01:33:50.916ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0df55fc2-6b39-4a29-9df0-909dac438014, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8762024-11-28T01:33:50.917ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 0df55fc2-6b39-4a29-9df0-909dac438014, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8772024-11-28T01:33:50.952ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 99301c48-601f-449e-8e73-447a87265d28, dataset_name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8782024-11-28T01:33:50.952ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 99301c48-601f-449e-8e73-447a87265d28, dataset_name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8792024-11-28T01:33:50.989ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 777baf9d-7c4a-4b11-b86c-ebb4af23f5f1, dataset_name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8802024-11-28T01:33:50.990ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 777baf9d-7c4a-4b11-b86c-ebb4af23f5f1, dataset_name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: Cockroach }, responder: ... }
8812024-11-28T01:33:51.021ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8822024-11-28T01:33:51.021ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/omicron-zones.json file = common/src/ledger.rs:177
8832024-11-28T01:33:51.021ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/omicron-zones.json file = common/src/ledger.rs:177
8842024-11-28T01:33:51.021ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8852024-11-28T01:33:51.107ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8862024-11-28T01:33:51.107ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8872024-11-28T01:33:51.107ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8882024-11-28T01:33:51.107ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8892024-11-28T01:33:51.107ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8902024-11-28T01:33:51.200ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28 file = illumos-utils/src/zone.rs:325
8912024-11-28T01:33:51.204ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246 file = illumos-utils/src/zone.rs:325
8922024-11-28T01:33:51.208ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014 file = illumos-utils/src/zone.rs:325
8932024-11-28T01:33:51.213ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1 file = illumos-utils/src/zone.rs:325
8942024-11-28T01:33:51.217ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9 file = illumos-utils/src/zone.rs:325
8952024-11-28T01:33:51.222ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28 file = illumos-utils/src/zone.rs:363
8962024-11-28T01:33:51.227ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246 file = illumos-utils/src/zone.rs:363
8972024-11-28T01:33:51.232ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014 file = illumos-utils/src/zone.rs:363
8982024-11-28T01:33:51.237ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1 file = illumos-utils/src/zone.rs:363
8992024-11-28T01:33:51.242ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9 file = illumos-utils/src/zone.rs:363
9002024-11-28T01:33:54.384ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService7'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='0df55fc2-6b39-4a29-9df0-909dac438014'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::5]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9012024-11-28T01:33:54.385ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014
9022024-11-28T01:33:54.398ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService6'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::3'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='057f14d6-21ef-4a1a-9443-743301729246'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::3]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9032024-11-28T01:33:54.398ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246
9042024-11-28T01:33:54.437ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService5'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='99301c48-601f-449e-8e73-447a87265d28'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::6]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9052024-11-28T01:33:54.437ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28
9062024-11-28T01:33:54.495ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService8'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='777baf9d-7c4a-4b11-b86c-ebb4af23f5f1'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::7]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9072024-11-28T01:33:54.495ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1
9082024-11-28T01:33:54.537ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014") failed: Property not found. retry in 58.75036ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014
9092024-11-28T01:33:54.545ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246") failed: Property not found. retry in 63.664048ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246
9102024-11-28T01:33:54.545ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService9'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::4'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='789268ca-0469-4ab5-abce-6379b90792b9'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::4]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
9112024-11-28T01:33:54.545ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9
9122024-11-28T01:33:54.580ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28") failed: Property not found. retry in 36.378337ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28
9132024-11-28T01:33:54.606ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014") failed: Property not found. retry in 82.101255ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014
9142024-11-28T01:33:54.727ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246") failed: Property not found. retry in 127.931544ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246
9152024-11-28T01:33:54.780ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28") failed: Property not found. retry in 54.661446ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28
9162024-11-28T01:33:55.250ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014") failed: Property not found. retry in 162.261613ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014
9172024-11-28T01:33:55.323ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28") failed: Property not found. retry in 118.678692ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28
9182024-11-28T01:33:55.506ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246") failed: Property not found. retry in 227.506435ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246
9192024-11-28T01:33:55.728ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014") failed: Property not found. retry in 248.691599ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014
9202024-11-28T01:33:55.826ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1") failed: Property not found. retry in 43.919238ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1
9212024-11-28T01:33:55.886ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28") failed: Property not found. retry in 331.362733ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28
9222024-11-28T01:33:55.978ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246") failed: Property not found. retry in 328.866472ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246
9232024-11-28T01:33:56.150ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1") failed: Property not found. retry in 51.017038ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1
9242024-11-28T01:33:56.266ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9") failed: Property not found. retry in 39.543051ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9
9252024-11-28T01:33:56.300ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28") failed: Property not found. retry in 455.911681ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28
9262024-11-28T01:33:56.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014") failed: Property not found. retry in 1.097983488s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014
9272024-11-28T01:33:56.344ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58694
9282024-11-28T01:33:56.344ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 227 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58694 req_id = 1925fc61-aa81-4372-91e2-e713d1204e50 response_code = 200 uri = /baseboard
9292024-11-28T01:33:56.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1") failed: Property not found. retry in 299.617229ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1
9302024-11-28T01:33:56.379ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9") failed: Property not found. retry in 58.096618ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9
9312024-11-28T01:33:56.400ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246") failed: Property not found. retry in 581.231261ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246
9322024-11-28T01:33:56.461ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9") failed: Property not found. retry in 229.75755ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9
9332024-11-28T01:33:56.741ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1") failed: Property not found. retry in 515.340796ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1
9342024-11-28T01:33:56.853ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9") failed: Property not found. retry in 358.175289ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9
9352024-11-28T01:33:56.990ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28") failed: Property not found. retry in 1.311484146s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28
9362024-11-28T01:33:57.033ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246") failed: Property not found. retry in 1.016230893s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246
9372024-11-28T01:33:57.226ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9") failed: Property not found. retry in 415.81032ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9
9382024-11-28T01:33:57.271ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1") failed: Property not found. retry in 798.347112ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1
9392024-11-28T01:33:57.442ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014") failed: Property not found. retry in 1.285195006s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014
9402024-11-28T01:33:57.655ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9") failed: Property not found. retry in 1.297306439s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9
9412024-11-28T01:33:58.062ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246") failed: Property not found. retry in 960.195714ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246
9422024-11-28T01:33:58.081ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1") failed: Property not found. retry in 1.226292137s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1
9432024-11-28T01:33:58.312ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28") failed: Property not found. retry in 974.600536ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28
9442024-11-28T01:33:58.740ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014") failed: Property not found. retry in 1.461287254s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014
9452024-11-28T01:33:58.965ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9") failed: Property not found. retry in 1.135592165s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9
9462024-11-28T01:33:59.035ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246") failed: Property not found. retry in 1.151075831s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246
9472024-11-28T01:33:59.301ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28") failed: Property not found. retry in 1.238290652s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28
9482024-11-28T01:33:59.321ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1") failed: Property not found. retry in 1.20716086s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1
9492024-11-28T01:33:59.751ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 243.872438ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
9502024-11-28T01:34:00.114ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9") failed: Property not found. retry in 1.375392594s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9
9512024-11-28T01:34:00.199ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246") failed: Property not found. retry in 603.553723ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246
9522024-11-28T01:34:00.214ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014") failed: Property not found. retry in 784.449682ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014
9532024-11-28T01:34:00.541ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1") failed: Property not found. retry in 780.393734ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1
9542024-11-28T01:34:00.550ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28") failed: Property not found. retry in 861.249409ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28
9552024-11-28T01:34:00.814ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246") failed: Property not found. retry in 1.251956638s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246
9562024-11-28T01:34:01.010ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014") failed: Property not found. retry in 728.574276ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014
9572024-11-28T01:34:01.332ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1") failed: Property not found. retry in 580.881547ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1
9582024-11-28T01:34:01.423ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28") failed: Property not found. retry in 884.888477ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28
9592024-11-28T01:34:01.500ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9") failed: Property not found. retry in 1.179263898s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9
9602024-11-28T01:34:01.749ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014") failed: Property not found. retry in 748.850445ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014
9612024-11-28T01:34:01.923ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1") failed: Property not found. retry in 764.63255ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1
9622024-11-28T01:34:02.077ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246") failed: Property not found. retry in 1.040956819s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246
9632024-11-28T01:34:02.317ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28") failed: Property not found. retry in 1.467996217s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28
9642024-11-28T01:34:02.507ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014") failed: Property not found. retry in 1.446237324s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014
9652024-11-28T01:34:02.689ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9") failed: Property not found. retry in 714.550709ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9
9662024-11-28T01:34:02.697ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1") failed: Property not found. retry in 899.217799ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1
9672024-11-28T01:34:03.126ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246") failed: Property not found. retry in 863.090105ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246
9682024-11-28T01:34:03.413ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9") failed: Property not found. retry in 711.41378ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9
9692024-11-28T01:34:03.607ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1") failed: Property not found. retry in 1.05958166s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1
9702024-11-28T01:34:03.795ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28") failed: Property not found. retry in 994.897199ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28
9712024-11-28T01:34:03.964ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014") failed: Property not found. retry in 674.230899ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014
9722024-11-28T01:34:03.998ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246") failed: Property not found. retry in 1.059232547s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246
9732024-11-28T01:34:04.134ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9") failed: Property not found. retry in 1.15138829s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9
9742024-11-28T01:34:04.648ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014") failed: Property not found. retry in 1.029882134s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014
9752024-11-28T01:34:04.676ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1") failed: Property not found. retry in 809.882177ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1
9762024-11-28T01:34:04.799ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28") failed: Property not found. retry in 888.161701ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28
9772024-11-28T01:34:05.066ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246") failed: Property not found. retry in 1.09369245s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246
9782024-11-28T01:34:05.293ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9") failed: Property not found. retry in 756.443117ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9
9792024-11-28T01:34:05.495ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1") failed: Property not found. retry in 1.13405793s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1
9802024-11-28T01:34:05.687ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014") failed: Property not found. retry in 774.345675ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014
9812024-11-28T01:34:05.696ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28") failed: Property not found. retry in 755.855911ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28
9822024-11-28T01:34:06.655ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_cockroachdb_99301c48-601f-449e-8e73-447a87265d28
9832024-11-28T01:34:06.655ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246
9842024-11-28T01:34:06.655ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_cockroachdb_0df55fc2-6b39-4a29-9df0-909dac438014
9852024-11-28T01:34:06.655ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_cockroachdb_777baf9d-7c4a-4b11-b86c-ebb4af23f5f1
9862024-11-28T01:34:06.655ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_cockroachdb_789268ca-0469-4ab5-abce-6379b90792b9
9872024-11-28T01:34:06.655ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
9882024-11-28T01:34:06.656ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
9892024-11-28T01:34:06.656ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 15674363 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:37535 req_id = 4bec2acf-b915-4b12-a4f8-5a626c80016e response_code = 204 uri = /omicron-zones
9902024-11-28T01:34:06.690ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:45666
9912024-11-28T01:34:06.720ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 30016 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:45666 req_id = a8d3f165-cc3e-49b4-8e7c-137872e5b17a response_code = 500 uri = /cockroachdb
9922024-11-28T01:34:06.720ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "a8d3f165-cc3e-49b4-8e7c-137872e5b17a", "content-length": "124", "date": "Thu, 28 Nov 2024 01:34:06 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "a8d3f165-cc3e-49b4-8e7c-137872e5b17a" } file = sled-agent/src/rack_setup/service.rs:1141 retry_after = 88.578274ms
9932024-11-28T01:34:06.837ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 27052 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:45666 req_id = e1ff2c93-fc0f-4e04-a156-0baac08cf629 response_code = 500 uri = /cockroachdb
9942024-11-28T01:34:06.837ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "e1ff2c93-fc0f-4e04-a156-0baac08cf629", "content-length": "124", "date": "Thu, 28 Nov 2024 01:34:06 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "e1ff2c93-fc0f-4e04-a156-0baac08cf629" } file = sled-agent/src/rack_setup/service.rs:1141 retry_after = 101.218577ms
9952024-11-28T01:34:06.967ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 28277 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:45666 req_id = 8df3d5ea-2dff-4718-980b-db35ca7c95ed response_code = 500 uri = /cockroachdb
9962024-11-28T01:34:06.967ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "8df3d5ea-2dff-4718-980b-db35ca7c95ed", "content-length": "124", "date": "Thu, 28 Nov 2024 01:34:06 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "8df3d5ea-2dff-4718-980b-db35ca7c95ed" } file = sled-agent/src/rack_setup/service.rs:1141 retry_after = 158.453143ms
9972024-11-28T01:34:07.156ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 29287 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:45666 req_id = d26aca97-a2be-4f38-9d2f-af8ba04e7819 response_code = 500 uri = /cockroachdb
9982024-11-28T01:34:07.156ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "d26aca97-a2be-4f38-9d2f-af8ba04e7819", "content-length": "124", "date": "Thu, 28 Nov 2024 01:34:06 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "d26aca97-a2be-4f38-9d2f-af8ba04e7819" } file = sled-agent/src/rack_setup/service.rs:1141 retry_after = 88.601723ms
9992024-11-28T01:34:07.272ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 27278 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:45666 req_id = 70ec4e67-0342-4b82-b598-67f1ccb694c1 response_code = 500 uri = /cockroachdb
10002024-11-28T01:34:07.273ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "70ec4e67-0342-4b82-b598-67f1ccb694c1", "content-length": "124", "date": "Thu, 28 Nov 2024 01:34:06 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "70ec4e67-0342-4b82-b598-67f1ccb694c1" } file = sled-agent/src/rack_setup/service.rs:1141 retry_after = 216.081787ms
10012024-11-28T01:34:07.515ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 25783 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:45666 req_id = def47b6a-02ce-49f7-be5b-4388fc3beff8 response_code = 500 uri = /cockroachdb
10022024-11-28T01:34:07.515ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "def47b6a-02ce-49f7-be5b-4388fc3beff8", "content-length": "124", "date": "Thu, 28 Nov 2024 01:34:07 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "def47b6a-02ce-49f7-be5b-4388fc3beff8" } file = sled-agent/src/rack_setup/service.rs:1141 retry_after = 309.100654ms
10032024-11-28T01:34:07.851ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 25761 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:45666 req_id = c242acd9-8aa3-4017-893d-b5fe62620edd response_code = 500 uri = /cockroachdb
10042024-11-28T01:34:07.851ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "c242acd9-8aa3-4017-893d-b5fe62620edd", "content-length": "124", "date": "Thu, 28 Nov 2024 01:34:07 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "c242acd9-8aa3-4017-893d-b5fe62620edd" } file = sled-agent/src/rack_setup/service.rs:1141 retry_after = 291.011416ms
10052024-11-28T01:34:08.170ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 26407 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:45666 req_id = 0dfa752f-a624-4671-9465-2c604ba392c2 response_code = 500 uri = /cockroachdb
10062024-11-28T01:34:08.170ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "0dfa752f-a624-4671-9465-2c604ba392c2", "content-length": "124", "date": "Thu, 28 Nov 2024 01:34:07 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "0dfa752f-a624-4671-9465-2c604ba392c2" } file = sled-agent/src/rack_setup/service.rs:1141 retry_after = 311.505824ms
10072024-11-28T01:34:08.508ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 26153 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:45666 req_id = b7fc6f48-f99c-4c4c-8213-23cc3b0928f8 response_code = 500 uri = /cockroachdb
10082024-11-28T01:34:08.509ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "b7fc6f48-f99c-4c4c-8213-23cc3b0928f8", "content-length": "124", "date": "Thu, 28 Nov 2024 01:34:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "b7fc6f48-f99c-4c4c-8213-23cc3b0928f8" } file = sled-agent/src/rack_setup/service.rs:1141 retry_after = 463.264111ms
10092024-11-28T01:34:09.001ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 26666 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:45666 req_id = 64f9ee67-9601-40e8-9610-cb8e84bddda9 response_code = 500 uri = /cockroachdb
10102024-11-28T01:34:09.001ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "64f9ee67-9601-40e8-9610-cb8e84bddda9", "content-length": "124", "date": "Thu, 28 Nov 2024 01:34:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "64f9ee67-9601-40e8-9610-cb8e84bddda9" } file = sled-agent/src/rack_setup/service.rs:1141 retry_after = 556.778342ms
10112024-11-28T01:34:09.587ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_057f14d6-21ef-4a1a-9443-743301729246 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 28793 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:45666 req_id = 58b28c90-9747-4b2d-ac77-6807e1ba1f8d response_code = 500 uri = /cockroachdb
10122024-11-28T01:34:09.587ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "58b28c90-9747-4b2d-ac77-6807e1ba1f8d", "content-length": "124", "date": "Thu, 28 Nov 2024 01:34:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "58b28c90-9747-4b2d-ac77-6807e1ba1f8d" } file = sled-agent/src/rack_setup/service.rs:1141 retry_after = 667.890675ms
10132024-11-28T01:34:10.289ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::3]:32221 file = sled-agent/src/services.rs:3743
10142024-11-28T01:34:22.523ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3761
10152024-11-28T01:34:26.145ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:3784
10162024-11-28T01:34:26.148ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 15888909 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:45666 req_id = f032605a-5818-4ac6-98cc-3018039f502e response_code = 204 uri = /cockroachdb
10172024-11-28T01:34:26.158ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 254.351245ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
10182024-11-28T01:34:26.190ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: 98733a1a-8c66-4d41-88fc-32826e30dc26 (service), filesystem_pool: Some(ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: e7a3f401-14e4-47b9-8060-4cbbfc840143 (service), filesystem_pool: Some(ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: f9751d6f-77c7-44d3-9105-87b0872eec3f (service), filesystem_pool: Some(ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 8ff676ac-4433-41a4-8ead-2afb6c570229 (service), filesystem_pool: Some(ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: dc5c6f99-a9da-4946-a74a-4accf50cbd91, kind: Service { id: 8ff676ac-4433-41a4-8ead-2afb6c570229 }, name: Name("ntp-8ff676ac-4433-41a4-8ead-2afb6c570229"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 184])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 057f14d6-21ef-4a1a-9443-743301729246 (service), filesystem_pool: Some(ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External } } } }, OmicronZoneConfig { id: 789268ca-0469-4ab5-abce-6379b90792b9 (service), filesystem_pool: Some(ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External } } } }, OmicronZoneConfig { id: 0df55fc2-6b39-4a29-9df0-909dac438014 (service), filesystem_pool: Some(ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External } } } }, OmicronZoneConfig { id: 99301c48-601f-449e-8e73-447a87265d28 (service), filesystem_pool: Some(ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 777baf9d-7c4a-4b11-b86c-ebb4af23f5f1 (service), filesystem_pool: Some(ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External } } } }, OmicronZoneConfig { id: b555dc2a-491b-4b5c-b5d7-24801f82c048 (service), filesystem_pool: Some(ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.1.164:53, nic: NetworkInterface { id: 7517eed1-3882-4d66-a665-f02c99641d63, kind: Service { id: b555dc2a-491b-4b5c-b5d7-24801f82c048 }, name: Name("external-dns-b555dc2a-491b-4b5c-b5d7-24801f82c048"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 105])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: 7b4111aa-1a34-450f-8a24-b1db940782bf (service), filesystem_pool: Some(ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.1.165:53, nic: NetworkInterface { id: bdbab4fe-05a5-497c-92d5-51ab355ee8a2, kind: Service { id: 7b4111aa-1a34-450f-8a24-b1db940782bf }, name: Name("external-dns-7b4111aa-1a34-450f-8a24-b1db940782bf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 193, 141])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: 0cdd90ac-34d6-41d9-bd04-18f3635adaf7 (service), filesystem_pool: Some(ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.1.166, nic: NetworkInterface { id: c047ac03-3e6b-4894-a737-123382b38a57, kind: Service { id: 0cdd90ac-34d6-41d9-bd04-18f3635adaf7 }, name: Name("nexus-0cdd90ac-34d6-41d9-bd04-18f3635adaf7"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 75369688-7093-4009-81fe-7452642ec8ea (service), filesystem_pool: Some(ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.1.167, nic: NetworkInterface { id: 57c41de5-57cc-4ec0-b2a4-ec18275db374, kind: Service { id: 75369688-7093-4009-81fe-7452642ec8ea }, name: Name("nexus-75369688-7093-4009-81fe-7452642ec8ea"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 80d50149-adaa-4067-b457-5cd986e26bb3 (service), filesystem_pool: Some(ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.1.168, nic: NetworkInterface { id: 5d808073-b6dd-4352-8286-483c26ccb36e, kind: Service { id: 80d50149-adaa-4067-b457-5cd986e26bb3 }, name: Name("nexus-80d50149-adaa-4067-b457-5cd986e26bb3"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 23eb10fc-92f9-426a-87ed-79c7f8d88f80 (service), filesystem_pool: Some(ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }), zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: 572caa46-b393-41e5-987b-55dda03a9b6b (service), filesystem_pool: Some(ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }), zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External } } } }, OmicronZoneConfig { id: 0a628f94-2c6e-4a3f-bb0a-424cacb9bda4 (service), filesystem_pool: Some(ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: 0006cef9-8dc5-45d7-a16c-4b17021af389 (service), filesystem_pool: Some(ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: fd6f17dc-9876-49a2-b1cd-691de7641038 (service), filesystem_pool: Some(ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: 84b9c0c5-4e7c-497b-bdf7-530f7affefbd (service), filesystem_pool: Some(ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External } } } }, OmicronZoneConfig { id: 4ef451f1-362e-4b8f-b326-cebe912a93f2 (service), filesystem_pool: Some(ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External } } } }, OmicronZoneConfig { id: 38fd655a-f8ea-4905-92f0-93d848854e08 (service), filesystem_pool: Some(ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External } } } }, OmicronZoneConfig { id: ed7bcb82-6a26-454d-9fa1-5649f99b7567 (service), filesystem_pool: Some(ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4ed6085e-23c7-4689-858f-d222cde8051e (service), filesystem_pool: Some(ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External } } } }, OmicronZoneConfig { id: c4aaac38-1e89-49ee-ac84-b46f390b0104 (service), filesystem_pool: Some(ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External } } } }, OmicronZoneConfig { id: c97a6484-b9e2-4c10-af20-0c345ef06c0d (service), filesystem_pool: Some(ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External } } } }, OmicronZoneConfig { id: 670e14e6-f825-4f64-b53b-9d0e2ac72917 (service), filesystem_pool: Some(ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External } } } }, OmicronZoneConfig { id: ca3e7ea2-a817-4a0e-b8d7-4b70764b832c (service), filesystem_pool: Some(ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:525 sled_address = [fd00:1122:3344:101::1]:12345
10192024-11-28T01:34:26.191ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:33459
10202024-11-28T01:34:26.192ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 98733a1a-8c66-4d41-88fc-32826e30dc26, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10212024-11-28T01:34:26.192ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 98733a1a-8c66-4d41-88fc-32826e30dc26, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: InternalDns }, responder: ... }
10222024-11-28T01:34:26.244ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e7a3f401-14e4-47b9-8060-4cbbfc840143, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10232024-11-28T01:34:26.244ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: e7a3f401-14e4-47b9-8060-4cbbfc840143, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10242024-11-28T01:34:26.286ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f9751d6f-77c7-44d3-9105-87b0872eec3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10252024-11-28T01:34:26.287ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: f9751d6f-77c7-44d3-9105-87b0872eec3f, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10262024-11-28T01:34:26.351ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 057f14d6-21ef-4a1a-9443-743301729246, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10272024-11-28T01:34:26.351ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 057f14d6-21ef-4a1a-9443-743301729246, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Cockroach }, responder: ... }
10282024-11-28T01:34:26.406ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:45987
10292024-11-28T01:34:26.407ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 190 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:45987 req_id = 45dd03c4-0992-4959-ba79-bbaba2f9b8e9 response_code = 200 uri = /baseboard
10302024-11-28T01:34:26.427ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 789268ca-0469-4ab5-abce-6379b90792b9, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10312024-11-28T01:34:26.427ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 789268ca-0469-4ab5-abce-6379b90792b9, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10322024-11-28T01:34:26.472ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0df55fc2-6b39-4a29-9df0-909dac438014, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10332024-11-28T01:34:26.472ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 0df55fc2-6b39-4a29-9df0-909dac438014, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10342024-11-28T01:34:26.519ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 99301c48-601f-449e-8e73-447a87265d28, dataset_name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10352024-11-28T01:34:26.519ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 99301c48-601f-449e-8e73-447a87265d28, dataset_name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10362024-11-28T01:34:26.562ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 777baf9d-7c4a-4b11-b86c-ebb4af23f5f1, dataset_name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10372024-11-28T01:34:26.562ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 777baf9d-7c4a-4b11-b86c-ebb4af23f5f1, dataset_name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: Cockroach }, responder: ... }
10382024-11-28T01:34:26.610ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b555dc2a-491b-4b5c-b5d7-24801f82c048, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10392024-11-28T01:34:26.610ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b555dc2a-491b-4b5c-b5d7-24801f82c048, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: ExternalDns }, responder: ... }
10402024-11-28T01:34:26.653ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 7b4111aa-1a34-450f-8a24-b1db940782bf, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10412024-11-28T01:34:26.653ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 7b4111aa-1a34-450f-8a24-b1db940782bf, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
10422024-11-28T01:34:26.698ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 572caa46-b393-41e5-987b-55dda03a9b6b, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Clickhouse }, responder: ... }) file = sled-storage/src/manager.rs:603
10432024-11-28T01:34:26.698ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 572caa46-b393-41e5-987b-55dda03a9b6b, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Clickhouse }, responder: ... }
10442024-11-28T01:34:26.739ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 84b9c0c5-4e7c-497b-bdf7-530f7affefbd, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10452024-11-28T01:34:26.739ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 84b9c0c5-4e7c-497b-bdf7-530f7affefbd, dataset_name: DatasetName { pool_name: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, kind: Crucible }, responder: ... }
10462024-11-28T01:34:26.782ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4ef451f1-362e-4b8f-b326-cebe912a93f2, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10472024-11-28T01:34:26.782ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 4ef451f1-362e-4b8f-b326-cebe912a93f2, dataset_name: DatasetName { pool_name: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, kind: Crucible }, responder: ... }
10482024-11-28T01:34:26.826ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 38fd655a-f8ea-4905-92f0-93d848854e08, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10492024-11-28T01:34:26.827ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 38fd655a-f8ea-4905-92f0-93d848854e08, dataset_name: DatasetName { pool_name: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, kind: Crucible }, responder: ... }
10502024-11-28T01:34:26.875ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed7bcb82-6a26-454d-9fa1-5649f99b7567, dataset_name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10512024-11-28T01:34:26.875ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ed7bcb82-6a26-454d-9fa1-5649f99b7567, dataset_name: DatasetName { pool_name: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, kind: Crucible }, responder: ... }
10522024-11-28T01:34:26.923ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4ed6085e-23c7-4689-858f-d222cde8051e, dataset_name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10532024-11-28T01:34:26.923ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 4ed6085e-23c7-4689-858f-d222cde8051e, dataset_name: DatasetName { pool_name: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, kind: Crucible }, responder: ... }
10542024-11-28T01:34:26.964ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c4aaac38-1e89-49ee-ac84-b46f390b0104, dataset_name: DatasetName { pool_name: ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10552024-11-28T01:34:26.964ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: c4aaac38-1e89-49ee-ac84-b46f390b0104, dataset_name: DatasetName { pool_name: ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External }, kind: Crucible }, responder: ... }
10562024-11-28T01:34:27.006ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c97a6484-b9e2-4c10-af20-0c345ef06c0d, dataset_name: DatasetName { pool_name: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10572024-11-28T01:34:27.006ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: c97a6484-b9e2-4c10-af20-0c345ef06c0d, dataset_name: DatasetName { pool_name: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, kind: Crucible }, responder: ... }
10582024-11-28T01:34:27.050ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 670e14e6-f825-4f64-b53b-9d0e2ac72917, dataset_name: DatasetName { pool_name: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10592024-11-28T01:34:27.050ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 670e14e6-f825-4f64-b53b-9d0e2ac72917, dataset_name: DatasetName { pool_name: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }, kind: Crucible }, responder: ... }
10602024-11-28T01:34:27.093ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ca3e7ea2-a817-4a0e-b8d7-4b70764b832c, dataset_name: DatasetName { pool_name: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10612024-11-28T01:34:27.093ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ca3e7ea2-a817-4a0e-b8d7-4b70764b832c, dataset_name: DatasetName { pool_name: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }, kind: Crucible }, responder: ... }
10622024-11-28T01:34:27.134ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10632024-11-28T01:34:27.134ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/omicron-zones.json file = common/src/ledger.rs:177
10642024-11-28T01:34:27.135ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/omicron-zones.json file = common/src/ledger.rs:177
10652024-11-28T01:34:27.135ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10662024-11-28T01:34:27.267ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
10672024-11-28T01:34:27.268ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10682024-11-28T01:34:27.269ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
10692024-11-28T01:34:27.331ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10702024-11-28T01:34:27.331ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10712024-11-28T01:34:27.331ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10722024-11-28T01:34:27.331ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10732024-11-28T01:34:27.334ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10742024-11-28T01:34:27.334ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10752024-11-28T01:34:27.334ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10762024-11-28T01:34:27.496ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
10772024-11-28T01:34:27.509ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
10782024-11-28T01:34:27.524ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
10792024-11-28T01:34:27.525ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10802024-11-28T01:34:27.525ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10812024-11-28T01:34:27.562ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10822024-11-28T01:34:27.562ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10832024-11-28T01:34:27.562ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10842024-11-28T01:34:27.562ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10852024-11-28T01:34:27.562ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104 file = illumos-utils/src/zone.rs:325
10862024-11-28T01:34:27.569ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80 file = illumos-utils/src/zone.rs:325
10872024-11-28T01:34:27.574ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b file = illumos-utils/src/zone.rs:325
10882024-11-28T01:34:27.580ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4 file = illumos-utils/src/zone.rs:325
10892024-11-28T01:34:27.585ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08 file = illumos-utils/src/zone.rs:325
10902024-11-28T01:34:27.591ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd file = illumos-utils/src/zone.rs:325
10912024-11-28T01:34:27.598ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e file = illumos-utils/src/zone.rs:325
10922024-11-28T01:34:27.603ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c file = illumos-utils/src/zone.rs:325
10932024-11-28T01:34:27.721ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104 file = illumos-utils/src/zone.rs:363
10942024-11-28T01:34:27.727ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80 file = illumos-utils/src/zone.rs:363
10952024-11-28T01:34:27.732ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b file = illumos-utils/src/zone.rs:363
10962024-11-28T01:34:27.737ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4 file = illumos-utils/src/zone.rs:363
10972024-11-28T01:34:27.742ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08 file = illumos-utils/src/zone.rs:363
10982024-11-28T01:34:27.747ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd file = illumos-utils/src/zone.rs:363
10992024-11-28T01:34:27.753ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e file = illumos-utils/src/zone.rs:363
11002024-11-28T01:34:27.759ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c file = illumos-utils/src/zone.rs:363
11012024-11-28T01:34:27.766ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038 file = illumos-utils/src/zone.rs:325
11022024-11-28T01:34:27.773ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567 file = illumos-utils/src/zone.rs:325
11032024-11-28T01:34:27.780ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d file = illumos-utils/src/zone.rs:325
11042024-11-28T01:34:27.786ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2 file = illumos-utils/src/zone.rs:325
11052024-11-28T01:34:27.792ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917 file = illumos-utils/src/zone.rs:325
11062024-11-28T01:34:27.799ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389 file = illumos-utils/src/zone.rs:325
11072024-11-28T01:34:27.879ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
11082024-11-28T01:34:27.879ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038 file = illumos-utils/src/zone.rs:363
11092024-11-28T01:34:27.891ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567 file = illumos-utils/src/zone.rs:363
11102024-11-28T01:34:27.902ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d file = illumos-utils/src/zone.rs:363
11112024-11-28T01:34:27.912ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2 file = illumos-utils/src/zone.rs:363
11122024-11-28T01:34:27.919ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917 file = illumos-utils/src/zone.rs:363
11132024-11-28T01:34:27.935ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389 file = illumos-utils/src/zone.rs:363
11142024-11-28T01:34:27.997ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
11152024-11-28T01:34:28.055ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
11162024-11-28T01:34:28.112ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
11172024-11-28T01:34:28.171ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
11182024-11-28T01:34:28.177ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
11192024-11-28T01:34:28.177ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
11202024-11-28T01:34:28.237ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte1", ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 193, 141]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
11212024-11-28T01:34:28.237ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1263 zone_type = external_dns
11222024-11-28T01:34:28.242ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
11232024-11-28T01:34:28.242ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
11242024-11-28T01:34:28.304ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte2", ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 213, 87]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
11252024-11-28T01:34:28.304ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1263 zone_type = nexus
11262024-11-28T01:34:28.308ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
11272024-11-28T01:34:28.309ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
11282024-11-28T01:34:28.364ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte3", ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 229, 184]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
11292024-11-28T01:34:28.364ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1263 zone_type = nexus
11302024-11-28T01:34:28.365ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
11312024-11-28T01:34:28.365ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
11322024-11-28T01:34:28.436ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte4", ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 177, 105]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
11332024-11-28T01:34:28.436ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1263 zone_type = external_dns
11342024-11-28T01:34:28.438ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
11352024-11-28T01:34:28.438ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
11362024-11-28T01:34:28.495ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte5", ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 201, 190]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
11372024-11-28T01:34:28.495ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1263 zone_type = nexus
11382024-11-28T01:34:28.505ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.165 file = clients/dpd-client/src/lib.rs:83
11392024-11-28T01:34:28.505ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.165 file = clients/dpd-client/src/lib.rs:90
11402024-11-28T01:34:28.506ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.167 file = clients/dpd-client/src/lib.rs:83
11412024-11-28T01:34:28.506ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.167 file = clients/dpd-client/src/lib.rs:90
11422024-11-28T01:34:28.506ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.164 file = clients/dpd-client/src/lib.rs:83
11432024-11-28T01:34:28.506ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.164 file = clients/dpd-client/src/lib.rs:90
11442024-11-28T01:34:28.506ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.166 file = clients/dpd-client/src/lib.rs:83
11452024-11-28T01:34:28.506ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.166 file = clients/dpd-client/src/lib.rs:90
11462024-11-28T01:34:28.507ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.168 file = clients/dpd-client/src/lib.rs:83
11472024-11-28T01:34:28.507ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.168 file = clients/dpd-client/src/lib.rs:90
11482024-11-28T01:34:28.508ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.165 file = clients/dpd-client/src/lib.rs:118
11492024-11-28T01:34:28.510ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.168 file = clients/dpd-client/src/lib.rs:118
11502024-11-28T01:34:28.512ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.167 file = clients/dpd-client/src/lib.rs:118
11512024-11-28T01:34:28.512ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
11522024-11-28T01:34:28.512ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
11532024-11-28T01:34:28.514ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.164 file = clients/dpd-client/src/lib.rs:118
11542024-11-28T01:34:28.517ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.166 file = clients/dpd-client/src/lib.rs:118
11552024-11-28T01:34:28.518ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
11562024-11-28T01:34:28.518ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
11572024-11-28T01:34:28.519ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
11582024-11-28T01:34:28.772ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf file = illumos-utils/src/zone.rs:325
11592024-11-28T01:34:28.787ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3 file = illumos-utils/src/zone.rs:325
11602024-11-28T01:34:28.800ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea file = illumos-utils/src/zone.rs:325
11612024-11-28T01:34:28.817ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048 file = illumos-utils/src/zone.rs:325
11622024-11-28T01:34:28.831ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7 file = illumos-utils/src/zone.rs:325
11632024-11-28T01:34:28.846ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf file = illumos-utils/src/zone.rs:363
11642024-11-28T01:34:28.863ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3 file = illumos-utils/src/zone.rs:363
11652024-11-28T01:34:28.875ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea file = illumos-utils/src/zone.rs:363
11662024-11-28T01:34:28.885ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048 file = illumos-utils/src/zone.rs:363
11672024-11-28T01:34:28.898ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7 file = illumos-utils/src/zone.rs:363
11682024-11-28T01:34:32.547ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService22'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::19'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_c7ca3a79-57f5-4e47-8807-03bd67966814/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::19'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='144a531d-84b1-43a6-92ac-1c56e2b44102'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11692024-11-28T01:34:32.547ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService14'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::14'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_0176efd8-6a11-46a9-8bcd-a97e447fcb50/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::14'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='446561fc-d3bf-493f-83a4-19c3384b15c0'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11702024-11-28T01:34:32.547ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService16'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::16'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_216077bb-f8f0-4fbf-8aa0-1db2249e65da/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::16'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='7b3e3b2d-a45f-4def-a1cd-6a1fd327fdcf'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11712024-11-28T01:34:32.548ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService17'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_75598bf6-d28b-4334-94db-398fc2a8b539/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1a'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='028c5ce8-66fd-4301-8288-ad83777f2a74'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11722024-11-28T01:34:32.548ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService21'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::13'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_8164ba35-0eff-436a-9b6b-80912419e510/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::13'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='80cac821-6f3f-464c-952e-c0f760d9555f'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11732024-11-28T01:34:32.548ZINFOSledAgent (ServiceManager): Booting oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
11742024-11-28T01:34:32.908ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService10'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::17'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_977ab5dc-c743-4525-b172-ecf8a9034abe/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::17'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='e82152fc-6695-4f6b-8939-9b7de292716d'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11752024-11-28T01:34:32.908ZINFOSledAgent (ServiceManager): Booting oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
11762024-11-28T01:34:32.916ZINFOSledAgent (ServiceManager): Booting oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
11772024-11-28T01:34:32.923ZINFOSledAgent (ServiceManager): Booting oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
11782024-11-28T01:34:32.931ZINFOSledAgent (ServiceManager): Booting oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
11792024-11-28T01:34:32.939ZINFOSledAgent (ServiceManager): Booting oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
11802024-11-28T01:34:33.238ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService24'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::9'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte1'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::9]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.6:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11812024-11-28T01:34:33.238ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
11822024-11-28T01:34:37.244ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 56.914138ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
11832024-11-28T01:34:37.253ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 29.649037ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
11842024-11-28T01:34:37.261ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 69.772978ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
11852024-11-28T01:34:37.271ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 73.801465ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
11862024-11-28T01:34:37.280ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 64.967555ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
11872024-11-28T01:34:37.289ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 70.629982ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
11882024-11-28T01:34:37.298ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 55.363737ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
11892024-11-28T01:34:37.306ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 128.898419ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
11902024-11-28T01:34:37.315ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 108.068571ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
11912024-11-28T01:34:37.342ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 85.870614ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
11922024-11-28T01:34:37.357ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 148.832494ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
11932024-11-28T01:34:37.367ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 59.818582ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
11942024-11-28T01:34:37.376ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 69.335631ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
11952024-11-28T01:34:37.386ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 51.628731ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
11962024-11-28T01:34:37.434ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 145.069704ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
11972024-11-28T01:34:37.447ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 262.803358ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
11982024-11-28T01:34:37.460ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 174.709251ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
11992024-11-28T01:34:37.472ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 250.293217ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
12002024-11-28T01:34:37.484ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 256.204186ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
12012024-11-28T01:34:37.497ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 279.387618ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
12022024-11-28T01:34:37.518ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 112.919621ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
12032024-11-28T01:34:37.592ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 547.626857ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
12042024-11-28T01:34:37.643ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 510.469793ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
12052024-11-28T01:34:37.654ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 313.433265ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
12062024-11-28T01:34:37.723ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 329.526144ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
12072024-11-28T01:34:37.732ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 466.767447ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
12082024-11-28T01:34:37.752ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 235.257132ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
12092024-11-28T01:34:37.787ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 453.553702ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
12102024-11-28T01:34:37.980ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 1.189439166s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
12112024-11-28T01:34:38.001ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 869.87721ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
12122024-11-28T01:34:38.063ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 629.900894ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
12132024-11-28T01:34:38.154ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 972.293766ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
12142024-11-28T01:34:38.167ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 875.349221ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
12152024-11-28T01:34:38.212ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 860.618729ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
12162024-11-28T01:34:38.254ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 900.640584ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
12172024-11-28T01:34:38.706ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 1.126563387s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
12182024-11-28T01:34:38.884ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 1.067217447s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
12192024-11-28T01:34:39.057ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 1.175617294s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
12202024-11-28T01:34:39.087ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 805.312659ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
12212024-11-28T01:34:39.141ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 1.090518532s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
12222024-11-28T01:34:39.168ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 1.333871075s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
12232024-11-28T01:34:39.182ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 852.052109ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
12242024-11-28T01:34:39.847ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 554.646028ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
12252024-11-28T01:34:39.906ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 1.267759342s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
12262024-11-28T01:34:39.966ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 515.671421ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
12272024-11-28T01:34:40.049ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 560.345053ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
12282024-11-28T01:34:40.099ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService20'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::18'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_2fcc59ee-335a-4c80-bbcf-e66263237ec3/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::18'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='98b170a3-6baf-468f-866e-b015e376f1f8'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12292024-11-28T01:34:40.099ZINFOSledAgent (ServiceManager): Booting oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
12302024-11-28T01:34:40.191ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService19'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::15'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_beaaa347-4478-48b4-b888-a5873241ded0/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::15'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='2f06f0fd-6400-4c01-a7bd-2779efb468e8'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12312024-11-28T01:34:40.191ZINFOSledAgent (ServiceManager): Booting oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
12322024-11-28T01:34:40.245ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 656.28221ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
12332024-11-28T01:34:40.258ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 1.390927633s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
12342024-11-28T01:34:40.393ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d") failed: Property not found. retry in 73.227875ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
12352024-11-28T01:34:40.415ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 644.830482ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
12362024-11-28T01:34:40.452ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 59.66197ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
12372024-11-28T01:34:40.478ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d") failed: Property not found. retry in 117.659856ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
12382024-11-28T01:34:40.494ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 1.300918807s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
12392024-11-28T01:34:40.514ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 1.090402534s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
12402024-11-28T01:34:40.525ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 57.723104ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
12412024-11-28T01:34:40.526ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService13'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::f'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::f'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12422024-11-28T01:34:40.526ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
12432024-11-28T01:34:40.561ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService23'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::10'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::10'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12442024-11-28T01:34:40.562ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
12452024-11-28T01:34:40.593ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 241.83074ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
12462024-11-28T01:34:40.605ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d") failed: Property not found. retry in 142.063083ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
12472024-11-28T01:34:40.663ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 1.227198857s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
12482024-11-28T01:34:40.783ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d") failed: Property not found. retry in 477.807505ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
12492024-11-28T01:34:40.857ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 317.857109ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
12502024-11-28T01:34:40.907ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 29.727463ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
12512024-11-28T01:34:40.919ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 573.44818ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
12522024-11-28T01:34:40.953ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 95.956246ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
12532024-11-28T01:34:40.964ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 43.748969ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
12542024-11-28T01:34:41.023ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 102.073842ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
12552024-11-28T01:34:41.194ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 216.851968ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
12562024-11-28T01:34:41.205ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 1.03509228s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
12572024-11-28T01:34:41.220ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 930.3127ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
12582024-11-28T01:34:41.232ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 252.781271ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
12592024-11-28T01:34:41.243ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 1.160413129s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
12602024-11-28T01:34:41.279ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d") failed: Property not found. retry in 1.112913642s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
12612024-11-28T01:34:41.279ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService25'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::c'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte2'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12622024-11-28T01:34:41.280ZINFOSledAgent (ServiceManager): Booting oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
12632024-11-28T01:34:41.424ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 248.008853ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
12642024-11-28T01:34:41.467ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 592.033947ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
12652024-11-28T01:34:41.488ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 25.757986ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
12662024-11-28T01:34:41.496ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 392.874126ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
12672024-11-28T01:34:41.503ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 765.78214ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
12682024-11-28T01:34:41.523ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 61.080298ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
12692024-11-28T01:34:41.595ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 186.050754ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
12702024-11-28T01:34:41.616ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 1.243045123s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
12712024-11-28T01:34:41.659ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 716.250084ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
12722024-11-28T01:34:41.682ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 719.117012ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
12732024-11-28T01:34:41.730ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService26'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte3'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12742024-11-28T01:34:41.730ZINFOSledAgent (ServiceManager): Booting oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
12752024-11-28T01:34:41.795ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 302.193253ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
12762024-11-28T01:34:41.804ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 1.008661695s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
12772024-11-28T01:34:41.844ZINFOSledAgent (ServiceManager): Profile for oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService11'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::d'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/oximeter"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="id" value='23eb10fc-92f9-426a-87ed-79c7f8d88f80'/> <propval type="astring" name="address" value='[fd00:1122:3344:101::d]:12223'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12782024-11-28T01:34:41.844ZINFOSledAgent (ServiceManager): Booting oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
12792024-11-28T01:34:41.894ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 57.655435ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
12802024-11-28T01:34:41.904ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 952.08969ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
12812024-11-28T01:34:41.913ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 952.238711ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
12822024-11-28T01:34:41.961ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 134.63931ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
12832024-11-28T01:34:42.012ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 35.395772ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
12842024-11-28T01:34:42.057ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 139.205953ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
12852024-11-28T01:34:42.207ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 453.238683ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
12862024-11-28T01:34:42.217ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 229.649118ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
12872024-11-28T01:34:42.227ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 239.03003ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
12882024-11-28T01:34:42.236ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 596.861502ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
12892024-11-28T01:34:42.250ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 522.675537ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
12902024-11-28T01:34:42.277ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 1.245845892s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
12912024-11-28T01:34:42.388ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 942.025436ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
12922024-11-28T01:34:42.405ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d") failed: Property not found. retry in 1.438130178s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
12932024-11-28T01:34:42.416ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 1.153106366s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
12942024-11-28T01:34:42.427ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 1.399525216s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
12952024-11-28T01:34:42.459ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 259.738832ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
12962024-11-28T01:34:42.479ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 465.595253ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
12972024-11-28T01:34:42.669ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 1.417384561s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
12982024-11-28T01:34:42.729ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 1.165778482s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
12992024-11-28T01:34:42.783ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 1.123843927s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
13002024-11-28T01:34:42.822ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 1.47939945s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
13012024-11-28T01:34:42.841ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 935.328595ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
13022024-11-28T01:34:42.864ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 1.324692537s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
13032024-11-28T01:34:42.873ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 742.075863ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
13042024-11-28T01:34:42.881ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 1.175382067s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
13052024-11-28T01:34:42.954ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 474.623793ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
13062024-11-28T01:34:43.340ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 1.410254754s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
13072024-11-28T01:34:43.439ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 1.157984219s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
13082024-11-28T01:34:43.533ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 1.36785493s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
13092024-11-28T01:34:43.579ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 574.138432ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
13102024-11-28T01:34:43.624ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 784.56337ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
13112024-11-28T01:34:43.786ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 1.056242829s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
13122024-11-28T01:34:43.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 765.959028ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
13132024-11-28T01:34:43.851ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d") failed: Property not found. retry in 937.211432ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
13142024-11-28T01:34:43.902ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 877.287265ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
13152024-11-28T01:34:43.916ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 1.296155208s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
13162024-11-28T01:34:44.066ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 674.54023ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
13172024-11-28T01:34:44.095ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 685.417567ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
13182024-11-28T01:34:44.162ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 1.344927059s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
13192024-11-28T01:34:44.198ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 842.092981ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
13202024-11-28T01:34:44.310ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 770.515321ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
13212024-11-28T01:34:44.417ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 1.243140053s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
13222024-11-28T01:34:44.834ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 896.954225ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
13232024-11-28T01:34:44.887ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 750.076639ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
13242024-11-28T01:34:44.954ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 706.164894ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
13252024-11-28T01:34:45.053ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 789.89294ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
13262024-11-28T01:34:45.081ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 617.811838ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
13272024-11-28T01:34:45.103ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 533.357768ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
13282024-11-28T01:34:45.141ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 1.092435225s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
13292024-11-28T01:34:45.164ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d") failed: Property not found. retry in 599.326636ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
13302024-11-28T01:34:45.182ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 1.350164894s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
13312024-11-28T01:34:45.194ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 976.254989ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
13322024-11-28T01:34:45.209ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 1.202622302s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
13332024-11-28T01:34:45.226ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 769.375943ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
13342024-11-28T01:34:45.519ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 815.10209ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
13352024-11-28T01:34:45.648ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 1.189671823s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
13362024-11-28T01:34:45.659ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 1.115696477s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
13372024-11-28T01:34:45.673ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 1.210572414s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
13382024-11-28T01:34:45.686ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 515.176833ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
13392024-11-28T01:34:45.711ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 773.802603ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
13402024-11-28T01:34:45.742ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 1.405166645s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
13412024-11-28T01:34:45.775ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d") failed: Property not found. retry in 1.469886661s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
13422024-11-28T01:34:45.856ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 1.491790928s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
13432024-11-28T01:34:46.009ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 1.125509632s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
13442024-11-28T01:34:46.186ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 952.845714ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
13452024-11-28T01:34:46.215ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 595.124154ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
13462024-11-28T01:34:46.246ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 913.975461ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
13472024-11-28T01:34:46.346ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 602.299626ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
13482024-11-28T01:34:46.424ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 998.877835ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
13492024-11-28T01:34:46.495ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 1.074949178s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
13502024-11-28T01:34:46.544ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 820.865298ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
13512024-11-28T01:34:46.573ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService27'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::8'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte4'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::8]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.5:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13522024-11-28T01:34:46.574ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048
13532024-11-28T01:34:46.790ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 1.090693408s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
13542024-11-28T01:34:46.809ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048") failed: Property not found. retry in 69.778254ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048
13552024-11-28T01:34:46.823ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 1.166078032s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
13562024-11-28T01:34:46.850ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 1.130189586s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
13572024-11-28T01:34:46.862ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService15'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::12'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::12'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='d75586d1-ffda-4ab7-858e-ea4910fc6034'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13582024-11-28T01:34:46.862ZINFOSledAgent (ServiceManager): Booting oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd
13592024-11-28T01:34:46.891ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048") failed: Property not found. retry in 101.665158ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048
13602024-11-28T01:34:46.906ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 571.03414ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
13612024-11-28T01:34:47.006ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 500.644549ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
13622024-11-28T01:34:47.027ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048") failed: Property not found. retry in 227.423859ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048
13632024-11-28T01:34:47.111ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService18'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::11'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::11'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13642024-11-28T01:34:47.111ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038
13652024-11-28T01:34:47.126ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd") failed: Property not found. retry in 67.162692ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd
13662024-11-28T01:34:47.148ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 680.356926ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
13672024-11-28T01:34:47.161ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 502.912106ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
13682024-11-28T01:34:47.172ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 1.366492684s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
13692024-11-28T01:34:47.184ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 1.318754568s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
13702024-11-28T01:34:47.204ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd") failed: Property not found. retry in 127.665183ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd
13712024-11-28T01:34:47.276ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d") failed: Property not found. retry in 1.178772747s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
13722024-11-28T01:34:47.301ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048") failed: Property not found. retry in 570.524821ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048
13732024-11-28T01:34:47.348ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd") failed: Property not found. retry in 245.616486ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd
13742024-11-28T01:34:47.359ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038") failed: Property not found. retry in 59.695081ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038
13752024-11-28T01:34:47.368ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 624.931665ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
13762024-11-28T01:34:47.379ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 720.655485ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
13772024-11-28T01:34:47.429ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038") failed: Property not found. retry in 52.332648ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038
13782024-11-28T01:34:47.440ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 881.700387ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
13792024-11-28T01:34:47.490ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 743.699962ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
13802024-11-28T01:34:47.504ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038") failed: Property not found. retry in 248.868498ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038
13812024-11-28T01:34:47.519ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 835.343075ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
13822024-11-28T01:34:47.583ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 1.368822989s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
13832024-11-28T01:34:47.605ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd") failed: Property not found. retry in 315.050483ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd
13842024-11-28T01:34:47.673ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 857.996786ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
13852024-11-28T01:34:47.766ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038") failed: Property not found. retry in 499.130761ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038
13862024-11-28T01:34:47.841ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 1.347017896s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
13872024-11-28T01:34:47.846ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService28'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte5'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13882024-11-28T01:34:47.846ZINFOSledAgent (ServiceManager): Booting oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7
13892024-11-28T01:34:47.884ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048") failed: Property not found. retry in 1.04953825s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048
13902024-11-28T01:34:47.897ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 692.935588ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
13912024-11-28T01:34:47.933ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd") failed: Property not found. retry in 959.186055ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd
13922024-11-28T01:34:47.996ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 1.07014495s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
13932024-11-28T01:34:48.009ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 909.392541ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
13942024-11-28T01:34:48.021ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 1.404087097s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
13952024-11-28T01:34:48.070ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7") failed: Property not found. retry in 71.282513ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7
13962024-11-28T01:34:48.111ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 1.018786723s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
13972024-11-28T01:34:48.156ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7") failed: Property not found. retry in 145.44168ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7
13982024-11-28T01:34:48.245ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 1.410157563s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
13992024-11-28T01:34:48.276ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038") failed: Property not found. retry in 506.696861ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038
14002024-11-28T01:34:48.313ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7") failed: Property not found. retry in 241.607398ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7
14012024-11-28T01:34:48.364ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 1.393549171s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
14022024-11-28T01:34:48.374ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 1.111726751s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
14032024-11-28T01:34:48.464ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d") failed: Property not found. retry in 1.49334054s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
14042024-11-28T01:34:48.512ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 1.470254326s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
14052024-11-28T01:34:48.541ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 1.283114352s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
14062024-11-28T01:34:48.551ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 1.449618955s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
14072024-11-28T01:34:48.564ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7") failed: Property not found. retry in 309.405666ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7
14082024-11-28T01:34:48.601ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 728.045245ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
14092024-11-28T01:34:48.793ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038") failed: Property not found. retry in 1.447919504s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038
14102024-11-28T01:34:48.885ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7") failed: Property not found. retry in 770.767897ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7
14112024-11-28T01:34:48.903ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd") failed: Property not found. retry in 758.550218ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd
14122024-11-28T01:34:48.931ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 540.084299ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
14132024-11-28T01:34:48.945ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048") failed: Property not found. retry in 1.107474029s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048
14142024-11-28T01:34:48.963ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 1.121225987s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
14152024-11-28T01:34:49.078ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 1.234500084s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
14162024-11-28T01:34:49.142ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 1.479076481s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
14172024-11-28T01:34:49.201ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 810.408257ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
14182024-11-28T01:34:49.344ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 1.306967522s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
14192024-11-28T01:34:49.438ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 940.843476ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
14202024-11-28T01:34:49.484ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 893.432127ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
14212024-11-28T01:34:49.499ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 782.630713ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
14222024-11-28T01:34:49.668ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 951.865183ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
14232024-11-28T01:34:49.681ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd") failed: Property not found. retry in 1.486908792s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd
14242024-11-28T01:34:49.692ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7") failed: Property not found. retry in 811.434778ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7
14252024-11-28T01:34:49.772ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 1.098058636s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
14262024-11-28T01:34:49.840ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 1.165117084s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
14272024-11-28T01:34:49.971ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d") failed: Property not found. retry in 1.10693451s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
14282024-11-28T01:34:49.995ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 643.259686ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
14292024-11-28T01:34:50.014ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 642.07777ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
14302024-11-28T01:34:50.026ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 861.245389ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
14312024-11-28T01:34:50.063ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048") failed: Property not found. retry in 1.301135156s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048
14322024-11-28T01:34:50.096ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 1.438508552s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
14332024-11-28T01:34:50.253ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038") failed: Property not found. retry in 545.486772ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038
14342024-11-28T01:34:50.295ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 1.106283826s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
14352024-11-28T01:34:50.327ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 1.478790689s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
14362024-11-28T01:34:50.388ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 885.073443ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
14372024-11-28T01:34:50.400ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 1.111656765s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
14382024-11-28T01:34:50.518ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7") failed: Property not found. retry in 659.419516ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7
14392024-11-28T01:34:50.631ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 679.966201ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
14402024-11-28T01:34:50.642ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 972.685954ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
14412024-11-28T01:34:50.654ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 1.030653005s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
14422024-11-28T01:34:50.667ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 736.519042ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
14432024-11-28T01:34:50.680ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 1.02195794s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
14442024-11-28T01:34:50.814ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038") failed: Property not found. retry in 871.322313ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038
14452024-11-28T01:34:50.888ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 897.525265ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
14462024-11-28T01:34:50.903ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 1.387183225s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
14472024-11-28T01:34:51.019ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 1.201429998s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
14482024-11-28T01:34:51.092ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d") failed: Property not found. retry in 1.202253342s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
14492024-11-28T01:34:51.183ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd") failed: Property not found. retry in 1.281241449s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd
14502024-11-28T01:34:51.198ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7") failed: Property not found. retry in 1.416902936s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7
14512024-11-28T01:34:51.286ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 730.636124ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
14522024-11-28T01:34:51.324ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 841.236318ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
14532024-11-28T01:34:51.378ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048") failed: Property not found. retry in 1.349523429s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048
14542024-11-28T01:34:51.414ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 580.036881ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
14552024-11-28T01:34:51.426ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 1.413118076s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
14562024-11-28T01:34:51.523ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 502.048374ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
14572024-11-28T01:34:51.547ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 697.225313ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
14582024-11-28T01:34:51.626ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 1.065268338s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
14592024-11-28T01:34:51.699ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 1.245782627s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
14602024-11-28T01:34:51.709ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038") failed: Property not found. retry in 1.135197801s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038
14612024-11-28T01:34:51.720ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 837.598235ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
14622024-11-28T01:34:51.798ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 532.618646ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
14632024-11-28T01:34:51.816ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 790.235049ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
14642024-11-28T01:34:52.007ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 924.396521ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
14652024-11-28T01:34:52.030ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 837.271213ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
14662024-11-28T01:34:52.043ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 1.358938716s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
14672024-11-28T01:34:52.177ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 1.026853444s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
14682024-11-28T01:34:52.232ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 656.819897ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
14692024-11-28T01:34:52.259ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 1.09686329s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
14702024-11-28T01:34:52.302ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 587.686823ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
14712024-11-28T01:34:52.311ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d") failed: Property not found. retry in 1.0745803s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
14722024-11-28T01:34:52.342ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 525.625379ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
14732024-11-28T01:34:52.479ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd") failed: Property not found. retry in 1.014477617s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd
14742024-11-28T01:34:52.503ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService12'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::e'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::e'/> <propval type="astring" name="listen_port" value='8123'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse-admin-single"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::e]:8888'/> <propval type="astring" name="ch_address" value='[fd00:1122:3344:101::e]:8123'/> <propval type="astring" name="ch_binary" value='/opt/oxide/clickhouse/clickhouse'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14752024-11-28T01:34:52.503ZINFOSledAgent (ServiceManager): Booting oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b
14762024-11-28T01:34:52.571ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 1.333331245s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
14772024-11-28T01:34:52.621ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 532.627618ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
14782024-11-28T01:34:52.636ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7") failed: Property not found. retry in 1.324772448s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7
14792024-11-28T01:34:52.708ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 973.433614ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
14802024-11-28T01:34:52.729ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b") failed: Property not found. retry in 58.855931ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b
14812024-11-28T01:34:52.741ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048") failed: Property not found. retry in 680.880916ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048
14822024-11-28T01:34:52.802ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b") failed: Property not found. retry in 50.140623ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b
14832024-11-28T01:34:53.090ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 912.202474ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
14842024-11-28T01:34:53.142ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b") failed: Property not found. retry in 127.903247ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b
14852024-11-28T01:34:53.156ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 640.019492ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
14862024-11-28T01:34:53.169ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 951.649399ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
14872024-11-28T01:34:53.184ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 840.838833ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
14882024-11-28T01:34:53.197ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 818.792909ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
14892024-11-28T01:34:53.209ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038") failed: Property not found. retry in 517.586082ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038
14902024-11-28T01:34:53.221ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 871.869095ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
14912024-11-28T01:34:53.235ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 1.346535704s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
14922024-11-28T01:34:53.248ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 1.365072084s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
14932024-11-28T01:34:53.263ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 810.499617ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
14942024-11-28T01:34:53.286ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b") failed: Property not found. retry in 373.145015ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b
14952024-11-28T01:34:53.372ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 1.095582952s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
14962024-11-28T01:34:53.398ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d") failed: Property not found. retry in 1.295888257s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
14972024-11-28T01:34:53.414ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 982.811642ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
14982024-11-28T01:34:53.433ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048") failed: Property not found. retry in 1.224712762s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048
14992024-11-28T01:34:53.508ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd") failed: Property not found. retry in 1.494937889s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd
15002024-11-28T01:34:53.673ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b") failed: Property not found. retry in 866.042683ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b
15012024-11-28T01:34:53.692ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 1.161015825s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
15022024-11-28T01:34:53.740ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038") failed: Property not found. retry in 1.355688229s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038
15032024-11-28T01:34:53.809ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 683.970423ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
15042024-11-28T01:34:53.914ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 949.744422ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
15052024-11-28T01:34:53.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7") failed: Property not found. retry in 1.007101767s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7
15062024-11-28T01:34:54.011ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 657.552194ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
15072024-11-28T01:34:54.027ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 742.46222ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
15082024-11-28T01:34:54.036ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 1.235416936s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
15092024-11-28T01:34:54.084ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 865.956534ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
15102024-11-28T01:34:54.102ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 517.539311ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
15112024-11-28T01:34:54.129ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 1.028800479s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
15122024-11-28T01:34:54.408ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 1.326731971s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
15132024-11-28T01:34:54.479ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 697.568955ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
15142024-11-28T01:34:54.503ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 674.833776ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
15152024-11-28T01:34:54.551ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b") failed: Property not found. retry in 1.036449452s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b
15162024-11-28T01:34:54.593ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 1.201226354s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
15172024-11-28T01:34:54.623ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 1.22677556s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
15182024-11-28T01:34:54.633ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 1.207124932s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
15192024-11-28T01:34:54.667ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048") failed: Property not found. retry in 1.21900699s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048
15202024-11-28T01:34:54.679ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 583.275356ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
15212024-11-28T01:34:54.704ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d") failed: Property not found. retry in 1.407743959s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
15222024-11-28T01:34:54.778ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 707.399481ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
15232024-11-28T01:34:54.864ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 953.305124ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
15242024-11-28T01:34:54.874ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 606.973493ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
15252024-11-28T01:34:54.967ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 838.663434ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
15262024-11-28T01:34:54.985ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7") failed: Property not found. retry in 1.484017096s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7
15272024-11-28T01:34:55.012ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd") failed: Property not found. retry in 689.556787ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd
15282024-11-28T01:34:55.106ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038") failed: Property not found. retry in 1.073120586s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038
15292024-11-28T01:34:55.171ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 1.091150492s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
15302024-11-28T01:34:55.187ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 977.740879ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
15312024-11-28T01:34:55.196ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 1.256823093s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
15322024-11-28T01:34:55.274ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 957.400309ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
15332024-11-28T01:34:55.283ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c") failed: Property not found. retry in 1.337347527s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
15342024-11-28T01:34:55.493ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389") failed: Property not found. retry in 1.315811379s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
15352024-11-28T01:34:55.502ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 588.545753ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
15362024-11-28T01:34:55.597ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b") failed: Property not found. retry in 769.497854ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b
15372024-11-28T01:34:55.711ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd") failed: Property not found. retry in 1.428347789s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd
15382024-11-28T01:34:55.746ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917") failed: Property not found. retry in 702.099928ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
15392024-11-28T01:34:55.804ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104") failed: Property not found. retry in 1.077715761s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
15402024-11-28T01:34:55.816ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80") failed: Property not found. retry in 1.379636198s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
15412024-11-28T01:34:55.827ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2") failed: Property not found. retry in 1.23642142s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
15422024-11-28T01:34:55.851ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567") failed: Property not found. retry in 666.541048ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
15432024-11-28T01:34:55.859ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3") failed: Property not found. retry in 770.309326ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
15442024-11-28T01:34:55.896ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048") failed: Property not found. retry in 831.511817ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048
15452024-11-28T01:34:56.101ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea") failed: Property not found. retry in 606.679339ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
15462024-11-28T01:34:56.122ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d") failed: Property not found. retry in 1.344151279s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
15472024-11-28T01:34:56.175ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08") failed: Property not found. retry in 1.070097273s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
15482024-11-28T01:34:56.191ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038") failed: Property not found. retry in 692.496272ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038
15492024-11-28T01:34:56.242ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf") failed: Property not found. retry in 556.390181ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
15502024-11-28T01:34:56.271ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e") failed: Property not found. retry in 849.930619ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
15512024-11-28T01:34:56.380ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b") failed: Property not found. retry in 1.292335189s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b
15522024-11-28T01:34:56.568ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:63925
15532024-11-28T01:34:56.572ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 782 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:63925 req_id = 58a5ae0c-4ba1-4257-b83c-6c78dda90fe8 response_code = 200 uri = /baseboard
15542024-11-28T01:34:56.574ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4") failed: Property not found. retry in 1.100361166s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
15552024-11-28T01:34:56.596ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7") failed: Property not found. retry in 602.502674ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7
15562024-11-28T01:34:56.747ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048") failed: Property not found. retry in 635.287697ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048
15572024-11-28T01:34:56.912ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038") failed: Property not found. retry in 755.108648ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038
15582024-11-28T01:34:57.160ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd") failed: Property not found. retry in 669.088904ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd
15592024-11-28T01:34:57.225ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 1.68160155s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
15602024-11-28T01:34:57.233ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7") failed: Property not found. retry in 1.078910816s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7
15612024-11-28T01:34:57.396ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048") failed: Property not found. retry in 1.226149742s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048
15622024-11-28T01:34:57.682ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038") failed: Property not found. retry in 706.380927ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038
15632024-11-28T01:34:57.695ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b") failed: Property not found. retry in 933.648096ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b
15642024-11-28T01:34:58.663ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b") failed: Property not found. retry in 1.320379745s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b
15652024-11-28T01:34:59.998ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b") failed: Property not found. retry in 668.287689ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b
15662024-11-28T01:35:00.683ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b") failed: Property not found. retry in 1.360209967s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b
15672024-11-28T01:35:02.054ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b") failed: Property not found. retry in 643.247013ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b
15682024-11-28T01:35:02.347ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 2.344803929s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
15692024-11-28T01:35:02.729ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_c4aaac38-1e89-49ee-ac84-b46f390b0104
15702024-11-28T01:35:02.729ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_external_dns_7b4111aa-1a34-450f-8a24-b1db940782bf
15712024-11-28T01:35:02.729ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_nexus_80d50149-adaa-4067-b457-5cd986e26bb3
15722024-11-28T01:35:02.729ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_oximeter_23eb10fc-92f9-426a-87ed-79c7f8d88f80
15732024-11-28T01:35:02.729ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_clickhouse_572caa46-b393-41e5-987b-55dda03a9b6b
15742024-11-28T01:35:02.729ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_pantry_0a628f94-2c6e-4a3f-bb0a-424cacb9bda4
15752024-11-28T01:35:02.730ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_38fd655a-f8ea-4905-92f0-93d848854e08
15762024-11-28T01:35:02.730ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_84b9c0c5-4e7c-497b-bdf7-530f7affefbd
15772024-11-28T01:35:02.730ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_4ed6085e-23c7-4689-858f-d222cde8051e
15782024-11-28T01:35:02.730ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_ca3e7ea2-a817-4a0e-b8d7-4b70764b832c
15792024-11-28T01:35:02.730ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_nexus_75369688-7093-4009-81fe-7452642ec8ea
15802024-11-28T01:35:02.730ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_external_dns_b555dc2a-491b-4b5c-b5d7-24801f82c048
15812024-11-28T01:35:02.730ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_pantry_fd6f17dc-9876-49a2-b1cd-691de7641038
15822024-11-28T01:35:02.730ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_ed7bcb82-6a26-454d-9fa1-5649f99b7567
15832024-11-28T01:35:02.730ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_nexus_0cdd90ac-34d6-41d9-bd04-18f3635adaf7
15842024-11-28T01:35:02.730ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_c97a6484-b9e2-4c10-af20-0c345ef06c0d
15852024-11-28T01:35:02.730ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_4ef451f1-362e-4b8f-b326-cebe912a93f2
15862024-11-28T01:35:02.730ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_670e14e6-f825-4f64-b53b-9d0e2ac72917
15872024-11-28T01:35:02.730ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_pantry_0006cef9-8dc5-45d7-a16c-4b17021af389
15882024-11-28T01:35:02.740ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
15892024-11-28T01:35:02.743ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
15902024-11-28T01:35:02.758ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36566773 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:33459 req_id = 7b60f59e-0c0f-45fb-bb71-41e089e134af response_code = 204 uri = /omicron-zones
15912024-11-28T01:35:02.763ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1435
15922024-11-28T01:35:02.772ZINFOSledAgent (RSS): Writing ledger to /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
15932024-11-28T01:35:02.772ZINFOSledAgent (RSS): Writing ledger to /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
15942024-11-28T01:35:02.790ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:774
15952024-11-28T01:35:02.794ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::a]:12221 file = sled-agent/src/rack_setup/service.rs:786
15962024-11-28T01:35:02.826ZINFOSledAgent (RSS): rack_network_config: RackNetworkConfigV2 { bfd: [], bgp: [], infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [ PortConfigV2 { addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.1.174, width: 24, }, ), vlan_id: None, }, ], autoneg: false, bgp_peers: [], lldp: None, port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.1.100, rib_priority: None, vlan_id: None, }, ], switch: Switch0, tx_eq: None, uplink_port_fec: Some( None, ), uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, } file = sled-agent/src/rack_setup/service.rs:1000
15972024-11-28T01:35:02.837ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/847c32d1-2da6-4de4-82bb-09f10249a7c3/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
15982024-11-28T01:35:02.901ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/847c32d1-2da6-4de4-82bb-09f10249a7c3/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
15992024-11-28T01:35:03.008ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/847c32d1-2da6-4de4-82bb-09f10249a7c3/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
16002024-11-28T01:35:03.136ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/847c32d1-2da6-4de4-82bb-09f10249a7c3/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
16012024-11-28T01:35:03.249ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/847c32d1-2da6-4de4-82bb-09f10249a7c3/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
16022024-11-28T01:35:03.387ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/847c32d1-2da6-4de4-82bb-09f10249a7c3/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
16032024-11-28T01:35:03.635ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/847c32d1-2da6-4de4-82bb-09f10249a7c3/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
16042024-11-28T01:35:03.832ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/847c32d1-2da6-4de4-82bb-09f10249a7c3/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
16052024-11-28T01:35:04.242ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/847c32d1-2da6-4de4-82bb-09f10249a7c3/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
16062024-11-28T01:35:04.591ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/847c32d1-2da6-4de4-82bb-09f10249a7c3/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
16072024-11-28T01:35:04.725ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 5.579983645s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
16082024-11-28T01:35:05.347ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/847c32d1-2da6-4de4-82bb-09f10249a7c3/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
16092024-11-28T01:35:06.101ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/847c32d1-2da6-4de4-82bb-09f10249a7c3/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
16102024-11-28T01:35:06.663ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/racks/847c32d1-2da6-4de4-82bb-09f10249a7c3/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1062
16112024-11-28T01:35:08.131ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "d15c2f1e-bb60-4b7d-b006-7a5e1bd5cee8", "content-length": "173", "date": "Thu, 28 Nov 2024 01:35:07 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"847c32d1-2da6-4de4-82bb-09f10249a7c3\"", request_id: "d15c2f1e-bb60-4b7d-b006-7a5e1bd5cee8" } file = sled-agent/src/rack_setup/service.rs:1062
16122024-11-28T01:35:09.090ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "26dca777-2249-4896-8066-4927fb13dcfc", "content-length": "173", "date": "Thu, 28 Nov 2024 01:35:09 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"847c32d1-2da6-4de4-82bb-09f10249a7c3\"", request_id: "26dca777-2249-4896-8066-4927fb13dcfc" } file = sled-agent/src/rack_setup/service.rs:1062
16132024-11-28T01:35:09.893ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "3895612a-81fb-4502-bfe7-1c4dee3b820d", "content-length": "173", "date": "Thu, 28 Nov 2024 01:35:09 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"847c32d1-2da6-4de4-82bb-09f10249a7c3\"", request_id: "3895612a-81fb-4502-bfe7-1c4dee3b820d" } file = sled-agent/src/rack_setup/service.rs:1062
16142024-11-28T01:35:14.270ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "77c9a5a3-5f97-4700-9207-bb1ba2ef39c1", "content-length": "173", "date": "Thu, 28 Nov 2024 01:35:14 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"12368a57-a643-445a-b32d-da10700bd20e\"", request_id: "77c9a5a3-5f97-4700-9207-bb1ba2ef39c1" } file = sled-agent/src/rack_setup/service.rs:1062
16152024-11-28T01:35:16.374ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "c6823a8f-383a-4170-a65e-5245a2b1ddd9", "content-length": "173", "date": "Thu, 28 Nov 2024 01:35:16 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"12368a57-a643-445a-b32d-da10700bd20e\"", request_id: "c6823a8f-383a-4170-a65e-5245a2b1ddd9" } file = sled-agent/src/rack_setup/service.rs:1062
16162024-11-28T01:35:18.491ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "72255a2f-4458-413d-8093-05668c786764", "content-length": "173", "date": "Thu, 28 Nov 2024 01:35:18 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"12368a57-a643-445a-b32d-da10700bd20e\"", request_id: "72255a2f-4458-413d-8093-05668c786764" } file = sled-agent/src/rack_setup/service.rs:1062
16172024-11-28T01:35:21.898ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "9037c38b-4897-48a2-91ee-f98300787b42", "content-length": "173", "date": "Thu, 28 Nov 2024 01:35:21 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"12368a57-a643-445a-b32d-da10700bd20e\"", request_id: "9037c38b-4897-48a2-91ee-f98300787b42" } file = sled-agent/src/rack_setup/service.rs:1062
16182024-11-28T01:35:24.179ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "1cf4645c-7d1c-4870-bff3-a5196844de99", "content-length": "173", "date": "Thu, 28 Nov 2024 01:35:23 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"12368a57-a643-445a-b32d-da10700bd20e\"", request_id: "1cf4645c-7d1c-4870-bff3-a5196844de99" } file = sled-agent/src/rack_setup/service.rs:1062
16192024-11-28T01:35:26.609ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:49738
16202024-11-28T01:35:26.609ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 28 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:49738 req_id = 5901a329-b423-4c71-993f-06f7c9b07d29 response_code = 200 uri = /baseboard
16212024-11-28T01:35:27.447ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "74814783-11aa-4ef7-a6f8-59f540e3cd36", "content-length": "173", "date": "Thu, 28 Nov 2024 01:35:27 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"12368a57-a643-445a-b32d-da10700bd20e\"", request_id: "74814783-11aa-4ef7-a6f8-59f540e3cd36" } file = sled-agent/src/rack_setup/service.rs:1062
16222024-11-28T01:35:33.026ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "7a9b6091-2d35-4691-ab3d-4b0764341064", "content-length": "173", "date": "Thu, 28 Nov 2024 01:35:32 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"12368a57-a643-445a-b32d-da10700bd20e\"", request_id: "7a9b6091-2d35-4691-ab3d-4b0764341064" } file = sled-agent/src/rack_setup/service.rs:1062
16232024-11-28T01:35:40.037ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "aae9a7b5-64b9-4e29-9247-445695a939f6", "content-length": "173", "date": "Thu, 28 Nov 2024 01:35:39 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"12368a57-a643-445a-b32d-da10700bd20e\"", request_id: "aae9a7b5-64b9-4e29-9247-445695a939f6" } file = sled-agent/src/rack_setup/service.rs:1062
16242024-11-28T01:35:48.581ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "551bc91d-c977-4929-9cdc-39779b417f93", "content-length": "173", "date": "Thu, 28 Nov 2024 01:35:48 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"12368a57-a643-445a-b32d-da10700bd20e\"", request_id: "551bc91d-c977-4929-9cdc-39779b417f93" } file = sled-agent/src/rack_setup/service.rs:1062
16252024-11-28T01:35:56.441ZINFOSledAgent (NexusNotifierTask): Successfully put SledAgentInfo to nexus file = sled-agent/src/nexus.rs:500 new_generation = 1 old_generation = 1 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
16262024-11-28T01:35:56.531ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45776
16272024-11-28T01:35:56.531ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
16282024-11-28T01:35:56.531ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
16292024-11-28T01:35:56.531ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/omicron-zones.json file = common/src/ledger.rs:177
16302024-11-28T01:35:56.531ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/omicron-zones.json file = common/src/ledger.rs:177
16312024-11-28T01:35:56.545ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16322024-11-28T01:35:56.546ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee
16332024-11-28T01:35:56.568ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16342024-11-28T01:35:56.568ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8164ba35-0eff-436a-9b6b-80912419e510
16352024-11-28T01:35:56.591ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16362024-11-28T01:35:56.591ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_0176efd8-6a11-46a9-8bcd-a97e447fcb50
16372024-11-28T01:35:56.612ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16382024-11-28T01:35:56.612ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_beaaa347-4478-48b4-b888-a5873241ded0
16392024-11-28T01:35:56.633ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16402024-11-28T01:35:56.633ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_216077bb-f8f0-4fbf-8aa0-1db2249e65da
16412024-11-28T01:35:56.648ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:35485
16422024-11-28T01:35:56.648ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:35485 req_id = 3806a96e-8d2f-4a84-ab00-84336d4ba596 response_code = 200 uri = /baseboard
16432024-11-28T01:35:56.654ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16442024-11-28T01:35:56.654ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_977ab5dc-c743-4525-b172-ecf8a9034abe
16452024-11-28T01:35:56.673ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16462024-11-28T01:35:56.673ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_2fcc59ee-335a-4c80-bbcf-e66263237ec3
16472024-11-28T01:35:56.693ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16482024-11-28T01:35:56.693ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c7ca3a79-57f5-4e47-8807-03bd67966814
16492024-11-28T01:35:56.712ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16502024-11-28T01:35:56.712ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_75598bf6-d28b-4334-94db-398fc2a8b539
16512024-11-28T01:35:56.724ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 193367 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45776 req_id = 560d6f41-30c9-49e3-89c7-5ce04f85ae1c response_code = 200 uri = /inventory
16522024-11-28T01:35:58.699ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35188
16532024-11-28T01:35:58.706ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 7517eed1-3882-4d66-a665-f02c99641d63, kind: Service { id: b555dc2a-491b-4b5c-b5d7-24801f82c048 }, name: Name("external-dns-b555dc2a-491b-4b5c-b5d7-24801f82c048"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 105])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bdbab4fe-05a5-497c-92d5-51ab355ee8a2, kind: Service { id: 7b4111aa-1a34-450f-8a24-b1db940782bf }, name: Name("external-dns-7b4111aa-1a34-450f-8a24-b1db940782bf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 193, 141])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 57c41de5-57cc-4ec0-b2a4-ec18275db374, kind: Service { id: 75369688-7093-4009-81fe-7452642ec8ea }, name: Name("nexus-75369688-7093-4009-81fe-7452642ec8ea"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5d808073-b6dd-4352-8286-483c26ccb36e, kind: Service { id: 80d50149-adaa-4067-b457-5cd986e26bb3 }, name: Name("nexus-80d50149-adaa-4067-b457-5cd986e26bb3"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c047ac03-3e6b-4894-a737-123382b38a57, kind: Service { id: 0cdd90ac-34d6-41d9-bd04-18f3635adaf7 }, name: Name("nexus-0cdd90ac-34d6-41d9-bd04-18f3635adaf7"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
16542024-11-28T01:35:58.706ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16552024-11-28T01:35:58.706ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16562024-11-28T01:35:58.706ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16572024-11-28T01:35:58.706ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16582024-11-28T01:35:58.706ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16592024-11-28T01:35:58.706ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
16602024-11-28T01:35:58.706ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6534 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35188 req_id = fd86dd06-5e30-43ea-b8c6-4b553463558c response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
16612024-11-28T01:35:58.707ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:1072
16622024-11-28T01:35:58.933ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60380
16632024-11-28T01:35:58.934ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
16642024-11-28T01:35:58.934ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
16652024-11-28T01:35:58.934ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/omicron-zones.json file = common/src/ledger.rs:177
16662024-11-28T01:35:58.934ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/omicron-zones.json file = common/src/ledger.rs:177
16672024-11-28T01:35:58.947ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16682024-11-28T01:35:58.947ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee
16692024-11-28T01:35:58.974ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16702024-11-28T01:35:58.974ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8164ba35-0eff-436a-9b6b-80912419e510
16712024-11-28T01:35:58.998ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16722024-11-28T01:35:58.998ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_0176efd8-6a11-46a9-8bcd-a97e447fcb50
16732024-11-28T01:35:59.021ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16742024-11-28T01:35:59.021ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_beaaa347-4478-48b4-b888-a5873241ded0
16752024-11-28T01:35:59.042ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16762024-11-28T01:35:59.042ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_216077bb-f8f0-4fbf-8aa0-1db2249e65da
16772024-11-28T01:35:59.062ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16782024-11-28T01:35:59.062ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_977ab5dc-c743-4525-b172-ecf8a9034abe
16792024-11-28T01:35:59.081ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16802024-11-28T01:35:59.081ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_2fcc59ee-335a-4c80-bbcf-e66263237ec3
16812024-11-28T01:35:59.103ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16822024-11-28T01:35:59.103ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c7ca3a79-57f5-4e47-8807-03bd67966814
16832024-11-28T01:35:59.126ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16842024-11-28T01:35:59.126ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_75598bf6-d28b-4334-94db-398fc2a8b539
16852024-11-28T01:35:59.139ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 205755 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60380 req_id = a9b3007b-259d-4cbb-b0e8-85e7d6a7f41d response_code = 200 uri = /inventory
16862024-11-28T01:35:59.597ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63632
16872024-11-28T01:35:59.597ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 7517eed1-3882-4d66-a665-f02c99641d63, kind: Service { id: b555dc2a-491b-4b5c-b5d7-24801f82c048 }, name: Name("external-dns-b555dc2a-491b-4b5c-b5d7-24801f82c048"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 105])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bdbab4fe-05a5-497c-92d5-51ab355ee8a2, kind: Service { id: 7b4111aa-1a34-450f-8a24-b1db940782bf }, name: Name("external-dns-7b4111aa-1a34-450f-8a24-b1db940782bf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 193, 141])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 57c41de5-57cc-4ec0-b2a4-ec18275db374, kind: Service { id: 75369688-7093-4009-81fe-7452642ec8ea }, name: Name("nexus-75369688-7093-4009-81fe-7452642ec8ea"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5d808073-b6dd-4352-8286-483c26ccb36e, kind: Service { id: 80d50149-adaa-4067-b457-5cd986e26bb3 }, name: Name("nexus-80d50149-adaa-4067-b457-5cd986e26bb3"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c047ac03-3e6b-4894-a737-123382b38a57, kind: Service { id: 0cdd90ac-34d6-41d9-bd04-18f3635adaf7 }, name: Name("nexus-0cdd90ac-34d6-41d9-bd04-18f3635adaf7"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
16882024-11-28T01:35:59.597ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16892024-11-28T01:35:59.597ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16902024-11-28T01:35:59.597ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16912024-11-28T01:35:59.597ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
16922024-11-28T01:35:59.597ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16932024-11-28T01:35:59.597ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
16942024-11-28T01:35:59.597ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 357 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63632 req_id = e72e5522-9777-4324-80cf-6b7e3385123c response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
16952024-11-28T01:35:59.662ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55295
16962024-11-28T01:35:59.662ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 7517eed1-3882-4d66-a665-f02c99641d63, kind: Service { id: b555dc2a-491b-4b5c-b5d7-24801f82c048 }, name: Name("external-dns-b555dc2a-491b-4b5c-b5d7-24801f82c048"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 105])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bdbab4fe-05a5-497c-92d5-51ab355ee8a2, kind: Service { id: 7b4111aa-1a34-450f-8a24-b1db940782bf }, name: Name("external-dns-7b4111aa-1a34-450f-8a24-b1db940782bf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 193, 141])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 57c41de5-57cc-4ec0-b2a4-ec18275db374, kind: Service { id: 75369688-7093-4009-81fe-7452642ec8ea }, name: Name("nexus-75369688-7093-4009-81fe-7452642ec8ea"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5d808073-b6dd-4352-8286-483c26ccb36e, kind: Service { id: 80d50149-adaa-4067-b457-5cd986e26bb3 }, name: Name("nexus-80d50149-adaa-4067-b457-5cd986e26bb3"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c047ac03-3e6b-4894-a737-123382b38a57, kind: Service { id: 0cdd90ac-34d6-41d9-bd04-18f3635adaf7 }, name: Name("nexus-0cdd90ac-34d6-41d9-bd04-18f3635adaf7"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
16972024-11-28T01:35:59.662ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16982024-11-28T01:35:59.662ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
16992024-11-28T01:35:59.662ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17002024-11-28T01:35:59.662ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17012024-11-28T01:35:59.662ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17022024-11-28T01:35:59.663ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
17032024-11-28T01:35:59.663ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 192 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:55295 req_id = 85dcd4c4-d0af-4ff0-86be-bb90e36464bb response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17042024-11-28T01:35:59.951ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57433
17052024-11-28T01:35:59.951ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 7517eed1-3882-4d66-a665-f02c99641d63, kind: Service { id: b555dc2a-491b-4b5c-b5d7-24801f82c048 }, name: Name("external-dns-b555dc2a-491b-4b5c-b5d7-24801f82c048"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 105])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bdbab4fe-05a5-497c-92d5-51ab355ee8a2, kind: Service { id: 7b4111aa-1a34-450f-8a24-b1db940782bf }, name: Name("external-dns-7b4111aa-1a34-450f-8a24-b1db940782bf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 193, 141])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 57c41de5-57cc-4ec0-b2a4-ec18275db374, kind: Service { id: 75369688-7093-4009-81fe-7452642ec8ea }, name: Name("nexus-75369688-7093-4009-81fe-7452642ec8ea"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5d808073-b6dd-4352-8286-483c26ccb36e, kind: Service { id: 80d50149-adaa-4067-b457-5cd986e26bb3 }, name: Name("nexus-80d50149-adaa-4067-b457-5cd986e26bb3"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c047ac03-3e6b-4894-a737-123382b38a57, kind: Service { id: 0cdd90ac-34d6-41d9-bd04-18f3635adaf7 }, name: Name("nexus-0cdd90ac-34d6-41d9-bd04-18f3635adaf7"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
17062024-11-28T01:35:59.951ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17072024-11-28T01:35:59.951ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17082024-11-28T01:35:59.951ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17092024-11-28T01:35:59.951ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17102024-11-28T01:35:59.951ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17112024-11-28T01:35:59.951ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
17122024-11-28T01:35:59.951ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 270 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:57433 req_id = 5795e6ff-d32a-439f-8604-1dd628be6444 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17132024-11-28T01:36:07.942ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44115
17142024-11-28T01:36:07.943ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 228 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44115 req_id = eb3d5ddd-a72d-46f5-b585-08cfae7cec95 response_code = 200 uri = /vpc-routes
17152024-11-28T01:36:07.970ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
17162024-11-28T01:36:07.970ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 994 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44115 req_id = 49c561c0-1bde-41e2-8af4-38768ee5d5d5 response_code = 204 uri = /eip-gateways
17172024-11-28T01:36:08.160ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1667 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44115 req_id = ea0c665d-383a-4b0d-86e3-a775e03b7208 response_code = 204 uri = /vpc-routes
17182024-11-28T01:36:12.228ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48122
17192024-11-28T01:36:12.228ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48122 req_id = 1f5ef39b-560e-48f5-b1a5-2521de1cf993 response_code = 200 uri = /vpc-routes
17202024-11-28T01:36:12.395ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40259
17212024-11-28T01:36:12.395ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40259 req_id = ab908470-a0c0-408c-9e9c-cb63e117521a response_code = 200 uri = /vpc-routes
17222024-11-28T01:36:12.452ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
17232024-11-28T01:36:12.453ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48122 req_id = 19be250c-9200-4ec0-b57d-5f6b4ce77dff response_code = 204 uri = /eip-gateways
17242024-11-28T01:36:12.500ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37677
17252024-11-28T01:36:12.501ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 730 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37677 req_id = 2c3ea817-e927-4654-a32f-fdeba020e054 response_code = 200 uri = /v2p
17262024-11-28T01:36:12.536ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56566
17272024-11-28T01:36:12.536ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56566 req_id = 877ee30b-c6bf-4e75-a36a-de1d94929b08 response_code = 200 uri = /v2p
17282024-11-28T01:36:12.543ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
17292024-11-28T01:36:12.543ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:40259 req_id = 106601c9-66d7-4136-b466-bb77428d50a0 response_code = 204 uri = /eip-gateways
17302024-11-28T01:36:12.641ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56686
17312024-11-28T01:36:12.641ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56686 req_id = 24bd740f-18fb-4fa7-95c8-b5fe81407009 response_code = 200 uri = /vpc-routes
17322024-11-28T01:36:12.653ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 282 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48122 req_id = e19fcbc3-a7f7-42b5-bf4b-d3b20e6276b3 response_code = 204 uri = /vpc-routes
17332024-11-28T01:36:12.690ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52215
17342024-11-28T01:36:12.690ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
17352024-11-28T01:36:12.776ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
17362024-11-28T01:36:12.851ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 161439 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:52215 req_id = 0b2b570c-86cb-489f-b0d9-00a9ad7e9b61 response_code = 204 uri = /switch-ports
17372024-11-28T01:36:12.851ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47553
17382024-11-28T01:36:12.852ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 307 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47553 req_id = 782f9ee9-1f15-48d0-8323-3374e0359e03 response_code = 200 uri = /v2p
17392024-11-28T01:36:12.852ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
17402024-11-28T01:36:12.852ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 247 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56686 req_id = 5a587b69-1b92-490d-b7cb-382c22339749 response_code = 204 uri = /eip-gateways
17412024-11-28T01:36:12.852ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 502 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:40259 req_id = d4de3a0d-cc47-4963-b690-9088861ca7bd response_code = 204 uri = /vpc-routes
17422024-11-28T01:36:12.852ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 555 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52215 req_id = e98e4e4d-4910-49aa-a79c-a2d532ec8dfa response_code = 200 uri = /network-bootstore-config
17432024-11-28T01:36:12.865ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56686 req_id = 99f1a8fd-37a9-4263-a34a-667c91ff26de response_code = 204 uri = /vpc-routes
17442024-11-28T01:36:12.877ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47626
17452024-11-28T01:36:12.877ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
17462024-11-28T01:36:12.922ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
17472024-11-28T01:36:12.984ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106403 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:47626 req_id = 167eb0af-5551-43cb-9a37-c1f105af5ee2 response_code = 204 uri = /switch-ports
17482024-11-28T01:36:12.984ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47657
17492024-11-28T01:36:12.984ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
17502024-11-28T01:36:12.984ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 257 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47626 req_id = 55b119ee-e2db-43bc-8ef5-9f71aa076c40 response_code = 200 uri = /network-bootstore-config
17512024-11-28T01:36:12.990ZINFOSledAgent (bootstore): Attempting to update network config with generation: 2, current_generation: 1 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-buskin-i86pc
17522024-11-28T01:36:12.991ZINFOSledAgent (bootstore): Writing ledger to /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
17532024-11-28T01:36:13.009ZINFOSledAgent (bootstore): Writing ledger to /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
17542024-11-28T01:36:13.011ZINFOSledAgent (bootstore): Broadcasting network config with generation 2 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-buskin-i86pc
17552024-11-28T01:36:13.011ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 27403 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52215 req_id = 31471e60-97c6-4901-bf29-2d9af58c3282 response_code = 204 uri = /network-bootstore-config
17562024-11-28T01:36:13.029ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
17572024-11-28T01:36:13.083ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99400 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:47657 req_id = 935721b9-d0a6-4565-b62f-05b3a7cee589 response_code = 204 uri = /switch-ports
17582024-11-28T01:36:13.084ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 160 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47657 req_id = c7a2df08-a6e2-4e28-9a21-40c5f3f09f36 response_code = 200 uri = /network-bootstore-config
17592024-11-28T01:36:26.682ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:56764
17602024-11-28T01:36:26.682ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 28 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:56764 req_id = e5c8406a-cd8a-46be-a410-bdaaa6d1b4ef response_code = 200 uri = /baseboard
17612024-11-28T01:36:42.192ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55790
17622024-11-28T01:36:42.192ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55790 req_id = f7cf760a-a8a2-4b2a-bbe7-3cad7ef35575 response_code = 200 uri = /vpc-routes
17632024-11-28T01:36:42.214ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
17642024-11-28T01:36:42.214ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55790 req_id = 2885202e-2954-4eb9-a614-4790720d533b response_code = 204 uri = /eip-gateways
17652024-11-28T01:36:42.227ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 185 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55790 req_id = 5a724055-cab5-4e8b-bd23-b74420b00316 response_code = 204 uri = /vpc-routes
17662024-11-28T01:36:42.308ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59704
17672024-11-28T01:36:42.308ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59704 req_id = fc94796f-7d80-4ffd-bc81-1e10f24b3d3e response_code = 200 uri = /vpc-routes
17682024-11-28T01:36:42.372ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39951
17692024-11-28T01:36:42.372ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 165 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39951 req_id = f6953031-d409-429f-9f23-e51891104304 response_code = 200 uri = /v2p
17702024-11-28T01:36:42.381ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
17712024-11-28T01:36:42.381ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 72 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59704 req_id = 81a36ddc-6f1a-41f5-9f00-ebb9fa059e5d response_code = 204 uri = /eip-gateways
17722024-11-28T01:36:42.393ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 174 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59704 req_id = cd8822a3-dca5-4180-b34c-b8c18c066f8b response_code = 204 uri = /vpc-routes
17732024-11-28T01:36:42.477ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38895
17742024-11-28T01:36:42.477ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38895 req_id = 8d90c218-3cd2-4684-ba92-cf35ab357a8e response_code = 200 uri = /vpc-routes
17752024-11-28T01:36:42.487ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59969
17762024-11-28T01:36:42.487ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59969 req_id = f63f3850-135b-4aa7-ace1-f9e20a6a79ca response_code = 200 uri = /v2p
17772024-11-28T01:36:42.527ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
17782024-11-28T01:36:42.527ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 75 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38895 req_id = 964546a2-24f8-4777-92c7-44100dbc8f95 response_code = 204 uri = /eip-gateways
17792024-11-28T01:36:42.537ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 208 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38895 req_id = 4b4474b8-3dff-4005-a8ba-2af5dff08b5e response_code = 204 uri = /vpc-routes
17802024-11-28T01:36:42.574ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58387
17812024-11-28T01:36:42.574ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
17822024-11-28T01:36:42.614ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
17832024-11-28T01:36:42.674ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99611 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:58387 req_id = 16726a02-20a8-417d-a620-bfd289f167a9 response_code = 204 uri = /switch-ports
17842024-11-28T01:36:42.674ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43100
17852024-11-28T01:36:42.674ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
17862024-11-28T01:36:42.678ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3958 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58387 req_id = e6e57e37-b52e-4c0c-8c72-fbcfdd937dd4 response_code = 200 uri = /network-bootstore-config
17872024-11-28T01:36:42.710ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
17882024-11-28T01:36:42.727ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33613
17892024-11-28T01:36:42.728ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 169 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33613 req_id = 7509837a-62e5-4ab4-8eff-a20c336079cb response_code = 200 uri = /v2p
17902024-11-28T01:36:42.768ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93614 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:43100 req_id = ecb1d415-d0f7-4100-be73-5f294782e762 response_code = 204 uri = /switch-ports
17912024-11-28T01:36:42.768ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43100 req_id = 2de49640-aba4-4cc0-8b20-b482a2a43a2f response_code = 200 uri = /network-bootstore-config
17922024-11-28T01:36:42.858ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47069
17932024-11-28T01:36:42.858ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
17942024-11-28T01:36:42.899ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
17952024-11-28T01:36:42.950ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91574 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:47069 req_id = 50b6f5c5-2420-4703-97e0-66b104c0d084 response_code = 204 uri = /switch-ports
17962024-11-28T01:36:42.950ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 203 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47069 req_id = ab5ac340-7325-4003-a306-a64a837c3862 response_code = 200 uri = /network-bootstore-config
17972024-11-28T01:36:56.716ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:44302
17982024-11-28T01:36:56.716ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 32 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:44302 req_id = 93937d90-88f1-4375-a2e6-41c99250bce0 response_code = 200 uri = /baseboard
17992024-11-28T01:37:12.394ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55619
18002024-11-28T01:37:12.394ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55619 req_id = 78408747-6fb3-40ff-8d6f-3824ffa180bd response_code = 200 uri = /vpc-routes
18012024-11-28T01:37:12.468ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
18022024-11-28T01:37:12.468ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55619 req_id = 896193bf-0d72-42f6-976a-3a60f8aa9320 response_code = 204 uri = /eip-gateways
18032024-11-28T01:37:12.480ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 242 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55619 req_id = 6f845590-be40-49f8-937b-8ce15b963619 response_code = 204 uri = /vpc-routes
18042024-11-28T01:37:12.489ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54948
18052024-11-28T01:37:12.489ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 34 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54948 req_id = 6f7e8acb-7030-401d-91de-19fd0c5ea0d3 response_code = 200 uri = /vpc-routes
18062024-11-28T01:37:12.556ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46826
18072024-11-28T01:37:12.557ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46826 req_id = c3a58b67-77dc-4e19-99c9-a2d2d1017e25 response_code = 200 uri = /v2p
18082024-11-28T01:37:12.600ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
18092024-11-28T01:37:12.600ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:54948 req_id = fb8e0ef4-e79f-4828-aef3-b01747919b8a response_code = 204 uri = /eip-gateways
18102024-11-28T01:37:12.618ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52759
18112024-11-28T01:37:12.618ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52759 req_id = e05475f5-3f59-456c-af74-6c8e2e481b49 response_code = 200 uri = /v2p
18122024-11-28T01:37:12.643ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46710
18132024-11-28T01:37:12.643ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
18142024-11-28T01:37:12.683ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
18152024-11-28T01:37:12.743ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99912 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:46710 req_id = 88555411-e343-4917-8f42-b36cc62ca7c6 response_code = 204 uri = /switch-ports
18162024-11-28T01:37:12.743ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40617
18172024-11-28T01:37:12.743ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40617 req_id = 75dca1de-d6e9-45e4-a492-d35d05cdcddb response_code = 200 uri = /vpc-routes
18182024-11-28T01:37:12.744ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 282 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46710 req_id = ababc062-8a80-47bc-9b75-057359409484 response_code = 200 uri = /network-bootstore-config
18192024-11-28T01:37:12.744ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 397 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:54948 req_id = dd986bd9-56c5-4395-bfea-249b197c8d82 response_code = 204 uri = /vpc-routes
18202024-11-28T01:37:12.758ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57808
18212024-11-28T01:37:12.758ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57808 req_id = e4eda789-b6e6-45c6-bbdd-80c3cf9a38a8 response_code = 200 uri = /v2p
18222024-11-28T01:37:12.797ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
18232024-11-28T01:37:12.797ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40617 req_id = a5a1fbec-7072-42af-bb09-b7a168bcce95 response_code = 204 uri = /eip-gateways
18242024-11-28T01:37:12.808ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 191 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40617 req_id = 69025779-a596-45fe-b290-b1f48118287b response_code = 204 uri = /vpc-routes
18252024-11-28T01:37:12.871ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39468
18262024-11-28T01:37:12.872ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
18272024-11-28T01:37:12.909ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
18282024-11-28T01:37:12.969ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97184 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:39468 req_id = cdfbe33f-5d3c-4967-9567-f073e8086819 response_code = 204 uri = /switch-ports
18292024-11-28T01:37:12.969ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36778
18302024-11-28T01:37:12.969ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
18312024-11-28T01:37:12.969ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 299 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39468 req_id = b220c5bf-ff92-47b4-b6ad-3cb37a68672b response_code = 200 uri = /network-bootstore-config
18322024-11-28T01:37:13.010ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
18332024-11-28T01:37:13.057ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88244 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:36778 req_id = 64ed5105-bfa0-436c-95e2-7c811e2c197b response_code = 204 uri = /switch-ports
18342024-11-28T01:37:13.058ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 158 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36778 req_id = 23619383-1539-4a28-874d-86a8668c6751 response_code = 200 uri = /network-bootstore-config
18352024-11-28T01:37:26.750ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:46913
18362024-11-28T01:37:26.750ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:46913 req_id = 66ee95e3-829f-4468-abad-800f6d1dd2e4 response_code = 200 uri = /baseboard
18372024-11-28T01:37:42.243ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56747
18382024-11-28T01:37:42.243ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 73 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56747 req_id = 5277c48a-0cba-429f-aef0-2d57aca1faa2 response_code = 200 uri = /vpc-routes
18392024-11-28T01:37:42.284ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
18402024-11-28T01:37:42.284ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 71 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:56747 req_id = a138e522-f1f1-4652-a1f3-ec167f0e1d72 response_code = 204 uri = /eip-gateways
18412024-11-28T01:37:42.346ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:65068
18422024-11-28T01:37:42.347ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 66 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:65068 req_id = 3795140e-8a80-4818-abf5-838e3e5c59c4 response_code = 200 uri = /vpc-routes
18432024-11-28T01:37:42.362ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56512
18442024-11-28T01:37:42.362ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 155 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56512 req_id = 7afee4e8-336b-49aa-bdda-33d58211f9dc response_code = 200 uri = /v2p
18452024-11-28T01:37:42.369ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 223 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:56747 req_id = 45dfc01c-1a4f-45b4-8bdc-08ecbf720442 response_code = 204 uri = /vpc-routes
18462024-11-28T01:37:42.403ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
18472024-11-28T01:37:42.403ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 75 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:65068 req_id = c1b19ed7-3943-45b6-b793-a99e13117cae response_code = 204 uri = /eip-gateways
18482024-11-28T01:37:42.438ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45809
18492024-11-28T01:37:42.438ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
18502024-11-28T01:37:42.480ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
18512024-11-28T01:37:42.541ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103443 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:45809 req_id = 4a102e29-916c-45eb-ae82-e84536e31fda response_code = 204 uri = /switch-ports
18522024-11-28T01:37:42.541ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57399
18532024-11-28T01:37:42.542ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46728
18542024-11-28T01:37:42.542ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60978
18552024-11-28T01:37:42.542ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 258 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45809 req_id = 7f72192f-c8e5-4a49-af22-b30218ef2fd9 response_code = 200 uri = /network-bootstore-config
18562024-11-28T01:37:42.542ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 380 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:65068 req_id = 1e742329-3081-466f-a662-2139d5c9bc7f response_code = 204 uri = /vpc-routes
18572024-11-28T01:37:42.542ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60978 req_id = 190e0c83-d84f-4365-b09f-97a9438437ba response_code = 200 uri = /vpc-routes
18582024-11-28T01:37:42.542ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 224 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57399 req_id = 21c61ad6-13ad-4d5a-85a4-b58c0d6255ef response_code = 200 uri = /v2p
18592024-11-28T01:37:42.543ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
18602024-11-28T01:37:42.554ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
18612024-11-28T01:37:42.554ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:60978 req_id = 1a7c2029-a6a4-4226-82a8-e05f3c9de6e0 response_code = 204 uri = /eip-gateways
18622024-11-28T01:37:42.579ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
18632024-11-28T01:37:42.607ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39912
18642024-11-28T01:37:42.607ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 284 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39912 req_id = 6ed1fd9d-5a0c-46de-b7da-344d62a2c369 response_code = 200 uri = /v2p
18652024-11-28T01:37:42.611ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 200 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:60978 req_id = dd4392d2-1587-4b70-ad93-9e031e8cc3ce response_code = 204 uri = /vpc-routes
18662024-11-28T01:37:42.640ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96793 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:46728 req_id = a1b81e43-9093-43a0-b37e-4f54f375241a response_code = 204 uri = /switch-ports
18672024-11-28T01:37:42.640ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 128 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46728 req_id = 3b32cb7b-05ca-4fd9-a16c-b1929385bb73 response_code = 200 uri = /network-bootstore-config
18682024-11-28T01:37:42.723ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40311
18692024-11-28T01:37:42.723ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
18702024-11-28T01:37:42.764ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
18712024-11-28T01:37:42.825ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101751 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:40311 req_id = a34570d8-a4a3-44ba-8b49-4c950c7282c9 response_code = 204 uri = /switch-ports
18722024-11-28T01:37:42.825ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40311 req_id = 929e4f9f-f462-484c-9a7b-cadc5124eb12 response_code = 200 uri = /network-bootstore-config
18732024-11-28T01:37:56.784ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:54410
18742024-11-28T01:37:56.784ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 28 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:54410 req_id = 8c5d1bc9-db8a-4a24-9b8a-a7c1960c6983 response_code = 200 uri = /baseboard
18752024-11-28T01:38:12.269ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38703
18762024-11-28T01:38:12.269ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38703 req_id = f901928e-deb9-47a6-9f96-13e12994932b response_code = 200 uri = /vpc-routes
18772024-11-28T01:38:12.387ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
18782024-11-28T01:38:12.387ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:38703 req_id = ac6b0a27-a272-4014-8e56-8b538a2502ed response_code = 204 uri = /eip-gateways
18792024-11-28T01:38:12.400ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 274 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:38703 req_id = 2217e0b3-c757-4430-8b17-e688a49d8563 response_code = 204 uri = /vpc-routes
18802024-11-28T01:38:12.402ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49628
18812024-11-28T01:38:12.402ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49628 req_id = bba41d44-7136-433c-8df8-953094448634 response_code = 200 uri = /vpc-routes
18822024-11-28T01:38:12.531ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
18832024-11-28T01:38:12.531ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:49628 req_id = 43672366-6d0c-45ad-82bf-5e306e42f372 response_code = 204 uri = /eip-gateways
18842024-11-28T01:38:12.558ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57594
18852024-11-28T01:38:12.558ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 132 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57594 req_id = 138d7c53-32ec-4263-bba0-00084ec2ad29 response_code = 200 uri = /v2p
18862024-11-28T01:38:12.585ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55826
18872024-11-28T01:38:12.585ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55826 req_id = 31ccc533-9a7a-4c56-96fd-e7b6393fe92c response_code = 200 uri = /v2p
18882024-11-28T01:38:12.595ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:49628 req_id = 8468d945-1c51-4d2e-8f20-67546f78b5c4 response_code = 204 uri = /vpc-routes
18892024-11-28T01:38:12.625ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43602
18902024-11-28T01:38:12.625ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
18912024-11-28T01:38:12.661ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
18922024-11-28T01:38:12.715ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90281 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:43602 req_id = 853b8de9-f80e-422d-b44f-fb2a7787bfd3 response_code = 204 uri = /switch-ports
18932024-11-28T01:38:12.716ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40062
18942024-11-28T01:38:12.716ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38397
18952024-11-28T01:38:12.716ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40062 req_id = cfcf2e92-34bb-45f1-816b-10f6b1a33101 response_code = 200 uri = /vpc-routes
18962024-11-28T01:38:12.716ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 264 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43602 req_id = 164cfa51-d2a8-406e-9d7f-95112b69ded5 response_code = 200 uri = /network-bootstore-config
18972024-11-28T01:38:12.716ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 186 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38397 req_id = 8a412c5d-5cc8-4810-8c0c-b7e2c31a05ff response_code = 200 uri = /v2p
18982024-11-28T01:38:12.737ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
18992024-11-28T01:38:12.737ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40062 req_id = e30cca21-c844-4834-ad55-e7f9b963d988 response_code = 204 uri = /eip-gateways
19002024-11-28T01:38:12.746ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40062 req_id = efdbfeb5-46fa-4ac7-a3cc-40e2c0040e9e response_code = 204 uri = /vpc-routes
19012024-11-28T01:38:12.752ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60683
19022024-11-28T01:38:12.752ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19032024-11-28T01:38:12.790ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19042024-11-28T01:38:12.848ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96625 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:60683 req_id = 978e1c8d-eb6e-4941-9266-005e3252bd24 response_code = 204 uri = /switch-ports
19052024-11-28T01:38:12.849ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65349
19062024-11-28T01:38:12.849ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19072024-11-28T01:38:12.849ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 182 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60683 req_id = 1f4da407-ea07-4ad4-bfb6-40a5473b796e response_code = 200 uri = /network-bootstore-config
19082024-11-28T01:38:12.881ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19092024-11-28T01:38:12.933ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83943 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:65349 req_id = 3f778955-cd00-4da2-a8b9-2bb58f12e5e4 response_code = 204 uri = /switch-ports
19102024-11-28T01:38:12.933ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65349 req_id = 2a8973b5-e00d-4a94-b134-7f17f57fe03a response_code = 200 uri = /network-bootstore-config
19112024-11-28T01:38:20.957ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
19122024-11-28T01:38:26.819ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43880
19132024-11-28T01:38:26.819ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43880 req_id = 054dddd6-b005-4bf8-acc3-ea06739a964e response_code = 200 uri = /baseboard
19142024-11-28T01:38:42.184ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35906
19152024-11-28T01:38:42.184ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35906 req_id = d83784b1-6cd1-4124-a3e7-b6e9e00e34fc response_code = 200 uri = /vpc-routes
19162024-11-28T01:38:42.240ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
19172024-11-28T01:38:42.240ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:35906 req_id = 7a844ad6-3fd7-4f3c-b7f6-f777a454658f response_code = 204 uri = /eip-gateways
19182024-11-28T01:38:42.241ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55778
19192024-11-28T01:38:42.241ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 39 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55778 req_id = 85221a54-8654-4d0a-993f-1513a5cb62da response_code = 200 uri = /vpc-routes
19202024-11-28T01:38:42.261ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
19212024-11-28T01:38:42.261ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55778 req_id = 69fb4724-9f27-4dba-9618-6bb5032bd62a response_code = 204 uri = /eip-gateways
19222024-11-28T01:38:42.270ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 179 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55778 req_id = 7b2783d6-19b6-48eb-8bd9-707542de65c0 response_code = 204 uri = /vpc-routes
19232024-11-28T01:38:42.349ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41288
19242024-11-28T01:38:42.349ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41288 req_id = a13a39b1-156d-4269-81c7-b4d736bc7f74 response_code = 200 uri = /v2p
19252024-11-28T01:38:42.359ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 157 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:35906 req_id = fd83ad17-28d4-4685-b39b-b06204d79adf response_code = 204 uri = /vpc-routes
19262024-11-28T01:38:42.374ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56142
19272024-11-28T01:38:42.374ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56142 req_id = 10f413df-ca2f-429d-a163-682b4c20b445 response_code = 200 uri = /v2p
19282024-11-28T01:38:42.382ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46119
19292024-11-28T01:38:42.382ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19302024-11-28T01:38:42.422ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19312024-11-28T01:38:42.497ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 115247 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:46119 req_id = 9a40339b-fda7-46f1-be86-b4934f1c4013 response_code = 204 uri = /switch-ports
19322024-11-28T01:38:42.498ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54662
19332024-11-28T01:38:42.498ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36096
19342024-11-28T01:38:42.498ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 281 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46119 req_id = e432bf69-4ace-4069-ba18-de5d9ecfbb02 response_code = 200 uri = /network-bootstore-config
19352024-11-28T01:38:42.504ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5693 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36096 req_id = 2522ba27-43a6-44f1-982c-d9ce8da1ac47 response_code = 200 uri = /vpc-routes
19362024-11-28T01:38:42.504ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19372024-11-28T01:38:42.547ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19382024-11-28T01:38:42.551ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
19392024-11-28T01:38:42.551ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 588 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36096 req_id = 6c27feaf-c21e-4235-8146-45474a03ee70 response_code = 204 uri = /eip-gateways
19402024-11-28T01:38:42.565ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 762 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36096 req_id = b7bdbd7d-e684-4dc3-9246-c29bf1aa1335 response_code = 204 uri = /vpc-routes
19412024-11-28T01:38:42.603ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58261
19422024-11-28T01:38:42.603ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 218 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58261 req_id = 4e116e2b-5232-4de1-847d-361826aebc89 response_code = 200 uri = /v2p
19432024-11-28T01:38:42.613ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 109766 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:54662 req_id = 8ee7c55b-2faf-483f-8014-695f53e1dc06 response_code = 204 uri = /switch-ports
19442024-11-28T01:38:42.614ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54662 req_id = c7781343-c63a-4da5-a363-63a17ea7390b response_code = 200 uri = /network-bootstore-config
19452024-11-28T01:38:42.618ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64582
19462024-11-28T01:38:42.619ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19472024-11-28T01:38:42.658ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19482024-11-28T01:38:42.717ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98778 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:64582 req_id = 54a8d3ab-335b-4c17-bcef-edfc1e8c0fbf response_code = 204 uri = /switch-ports
19492024-11-28T01:38:42.718ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64582 req_id = 274b9955-e30e-4189-bea7-c42222e545f2 response_code = 200 uri = /network-bootstore-config
19502024-11-28T01:38:56.853ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43909
19512024-11-28T01:38:56.853ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 62 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43909 req_id = ff28f8f6-18f4-4f3a-8451-f2de1537a902 response_code = 200 uri = /baseboard
19522024-11-28T01:39:12.349ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33864
19532024-11-28T01:39:12.349ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33864 req_id = a2bcb83d-066a-4d2b-9123-a631cfe5a1d3 response_code = 200 uri = /vpc-routes
19542024-11-28T01:39:12.449ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
19552024-11-28T01:39:12.450ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:33864 req_id = a54e8633-8770-4bc5-afa0-cb0369347618 response_code = 204 uri = /eip-gateways
19562024-11-28T01:39:12.460ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 236 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:33864 req_id = 3a29d57b-e016-4e21-ba60-02df51b26a13 response_code = 204 uri = /vpc-routes
19572024-11-28T01:39:12.511ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51288
19582024-11-28T01:39:12.511ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51288 req_id = 91321186-7da0-49c6-aa98-660b62962294 response_code = 200 uri = /vpc-routes
19592024-11-28T01:39:12.538ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
19602024-11-28T01:39:12.538ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:51288 req_id = 0694129f-47f1-43f3-b1b8-867ddd0633f6 response_code = 204 uri = /eip-gateways
19612024-11-28T01:39:12.551ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 177 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:51288 req_id = 82b0cb86-ed21-40a9-9503-6cec41972b2d response_code = 204 uri = /vpc-routes
19622024-11-28T01:39:12.591ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44536
19632024-11-28T01:39:12.592ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 150 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44536 req_id = bef0c823-7d88-4b47-ac08-8e89b639de3e response_code = 200 uri = /v2p
19642024-11-28T01:39:12.603ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37656
19652024-11-28T01:39:12.603ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19662024-11-28T01:39:12.644ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19672024-11-28T01:39:12.702ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99563 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:37656 req_id = 75275da4-106c-4ec4-b29d-9ab63d7e3042 response_code = 204 uri = /switch-ports
19682024-11-28T01:39:12.702ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33570
19692024-11-28T01:39:12.703ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 138 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37656 req_id = d71e7e97-cdeb-43a0-9ff6-577dbf89a55c response_code = 200 uri = /network-bootstore-config
19702024-11-28T01:39:12.703ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39134
19712024-11-28T01:39:12.703ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19722024-11-28T01:39:12.707ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4060 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39134 req_id = d03a9e9c-06bc-470d-af70-dcd6663398d7 response_code = 200 uri = /v2p
19732024-11-28T01:39:12.726ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56469
19742024-11-28T01:39:12.726ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56469 req_id = 0913ace5-7218-4818-83ab-35adcc37f1a3 response_code = 200 uri = /vpc-routes
19752024-11-28T01:39:12.740ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19762024-11-28T01:39:12.773ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
19772024-11-28T01:39:12.773ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56469 req_id = 0a1c84a3-a254-42b3-8862-f52137a96948 response_code = 204 uri = /eip-gateways
19782024-11-28T01:39:12.786ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 939 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56469 req_id = 1f7b0e5d-eed1-48e2-b31b-3d9d86e01480 response_code = 204 uri = /vpc-routes
19792024-11-28T01:39:12.808ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104986 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:33570 req_id = 4fc50ca2-cd9c-4308-8f30-2c0e5c4756f2 response_code = 204 uri = /switch-ports
19802024-11-28T01:39:12.809ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 147 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33570 req_id = 6d6e459c-6d4d-4a59-aa1f-1799a1dfbee4 response_code = 200 uri = /network-bootstore-config
19812024-11-28T01:39:12.836ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57773
19822024-11-28T01:39:12.836ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57773 req_id = d9005114-7273-46e7-871f-165fa705f515 response_code = 200 uri = /v2p
19832024-11-28T01:39:12.908ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43293
19842024-11-28T01:39:12.908ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19852024-11-28T01:39:12.943ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19862024-11-28T01:39:12.997ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88957 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:43293 req_id = 1ef7d978-af53-4a59-ae03-8591b51e1cc0 response_code = 204 uri = /switch-ports
19872024-11-28T01:39:12.997ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43293 req_id = 5317821a-99db-467f-b7e9-da258b14418c response_code = 200 uri = /network-bootstore-config
19882024-11-28T01:39:26.886ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:44143
19892024-11-28T01:39:26.886ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 28 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:44143 req_id = 6a6179e0-e5dd-4d43-af9e-f4f20ab09e35 response_code = 200 uri = /baseboard
19902024-11-28T01:39:42.183ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58599
19912024-11-28T01:39:42.183ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58599 req_id = 04fc94e1-0dca-459c-8872-29c76885b7ab response_code = 200 uri = /vpc-routes
19922024-11-28T01:39:42.201ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
19932024-11-28T01:39:42.201ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58599 req_id = 2ca7f12b-d103-4646-b462-4d3f1a0724aa response_code = 204 uri = /eip-gateways
19942024-11-28T01:39:42.240ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46748
19952024-11-28T01:39:42.241ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 139 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46748 req_id = f0d0cbde-5892-494b-98d1-1443cf9bda5c response_code = 200 uri = /v2p
19962024-11-28T01:39:42.253ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 256 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58599 req_id = 3219886d-7b30-490f-9271-4a36af861896 response_code = 204 uri = /vpc-routes
19972024-11-28T01:39:42.281ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55996
19982024-11-28T01:39:42.281ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19992024-11-28T01:39:42.328ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20002024-11-28T01:39:42.420ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 138184 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:55996 req_id = f46eae7b-87d5-4f77-8029-d1c918cd283e response_code = 204 uri = /switch-ports
20012024-11-28T01:39:42.420ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63147
20022024-11-28T01:39:42.420ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54141
20032024-11-28T01:39:42.420ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41283
20042024-11-28T01:39:42.420ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 171 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63147 req_id = efae88f8-7d15-48cd-8b1e-93eedfafe663 response_code = 200 uri = /vpc-routes
20052024-11-28T01:39:42.420ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 219 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41283 req_id = 0fab94f0-c071-4e96-8726-168fe8567e8f response_code = 200 uri = /v2p
20062024-11-28T01:39:42.420ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 255 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55996 req_id = 7ab0c3f8-7682-4fd6-9178-46838f2e1e0c response_code = 200 uri = /network-bootstore-config
20072024-11-28T01:39:42.420ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20082024-11-28T01:39:42.440ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
20092024-11-28T01:39:42.441ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 556 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63147 req_id = 5686b159-33e7-45b8-b573-1894599d7f05 response_code = 204 uri = /eip-gateways
20102024-11-28T01:39:42.454ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 595 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63147 req_id = a6efe4fa-b00f-45f6-819d-f5668c0ac315 response_code = 204 uri = /vpc-routes
20112024-11-28T01:39:42.461ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20122024-11-28T01:39:42.515ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95278 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:54141 req_id = d413b9ea-c759-421b-8d27-4c3e9e55149c response_code = 204 uri = /switch-ports
20132024-11-28T01:39:42.516ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 202 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54141 req_id = c14ef205-9018-453d-8490-95d02e472596 response_code = 200 uri = /network-bootstore-config
20142024-11-28T01:39:42.528ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35590
20152024-11-28T01:39:42.528ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35590 req_id = af7bc52a-b9c2-4a87-98c4-e2ee590fd78c response_code = 200 uri = /vpc-routes
20162024-11-28T01:39:42.579ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
20172024-11-28T01:39:42.579ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35590 req_id = 95453954-f3cf-4a83-8370-9e3de51c9104 response_code = 204 uri = /eip-gateways
20182024-11-28T01:39:42.588ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 257 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35590 req_id = 3d511fc7-a8c1-4618-b126-abb812db3ed4 response_code = 204 uri = /vpc-routes
20192024-11-28T01:39:42.632ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33090
20202024-11-28T01:39:42.632ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 120 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33090 req_id = 5feba975-da75-4dc5-ae2d-db415fd0e646 response_code = 200 uri = /v2p
20212024-11-28T01:39:42.752ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49755
20222024-11-28T01:39:42.752ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20232024-11-28T01:39:42.792ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20242024-11-28T01:39:42.848ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96323 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:49755 req_id = 30f7a947-1e97-4ce7-a4f1-e426bc88daa7 response_code = 204 uri = /switch-ports
20252024-11-28T01:39:42.849ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49755 req_id = 70ce4491-2a15-47c2-a8de-5f39ae3bb3b5 response_code = 200 uri = /network-bootstore-config
20262024-11-28T01:39:56.919ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:49590
20272024-11-28T01:39:56.919ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 33 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:49590 req_id = d80b22d7-f402-430e-99dd-eddd6e4ff0da response_code = 200 uri = /baseboard
20282024-11-28T01:40:12.266ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42288
20292024-11-28T01:40:12.266ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42288 req_id = b9752c85-f461-4164-8c71-a1d5b713f8c1 response_code = 200 uri = /vpc-routes
20302024-11-28T01:40:12.324ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58463
20312024-11-28T01:40:12.324ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58463 req_id = 03aa5065-d1ba-4152-8f20-9b963575bfa3 response_code = 200 uri = /vpc-routes
20322024-11-28T01:40:12.346ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
20332024-11-28T01:40:12.346ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42288 req_id = bea7647e-cc10-453f-8e39-d64d68bcafc4 response_code = 204 uri = /eip-gateways
20342024-11-28T01:40:12.374ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 282 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42288 req_id = 87ab9d12-526e-47e3-b800-125b62da0c22 response_code = 204 uri = /vpc-routes
20352024-11-28T01:40:12.449ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
20362024-11-28T01:40:12.449ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58463 req_id = 4ae51ff9-db9d-48fc-9832-a4772a906a9d response_code = 204 uri = /eip-gateways
20372024-11-28T01:40:12.459ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58463 req_id = dd20cf00-c3b1-41d8-8145-a1f7ef90e9bb response_code = 204 uri = /vpc-routes
20382024-11-28T01:40:12.537ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40117
20392024-11-28T01:40:12.537ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 144 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40117 req_id = f10c2f3d-9557-488f-a2cb-495c330bd253 response_code = 200 uri = /v2p
20402024-11-28T01:40:12.538ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33473
20412024-11-28T01:40:12.538ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33473 req_id = 1e0e014f-e696-4cc0-87de-a23ff5d8fb31 response_code = 200 uri = /v2p
20422024-11-28T01:40:12.576ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35840
20432024-11-28T01:40:12.576ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20442024-11-28T01:40:12.628ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20452024-11-28T01:40:12.713ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 136713 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:35840 req_id = 3678fd0c-4caf-4717-afc2-87b760cf972e response_code = 204 uri = /switch-ports
20462024-11-28T01:40:12.713ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53835
20472024-11-28T01:40:12.713ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48701
20482024-11-28T01:40:12.713ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45190
20492024-11-28T01:40:12.713ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20502024-11-28T01:40:12.713ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 284 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35840 req_id = 016b2bdc-c941-4364-bec1-2c5c4712dcb4 response_code = 200 uri = /network-bootstore-config
20512024-11-28T01:40:12.713ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 183 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48701 req_id = 46a5ce5b-440a-49b5-a02a-cc531de9d805 response_code = 200 uri = /vpc-routes
20522024-11-28T01:40:12.725ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 7517eed1-3882-4d66-a665-f02c99641d63, kind: Service { id: b555dc2a-491b-4b5c-b5d7-24801f82c048 }, name: Name("external-dns-b555dc2a-491b-4b5c-b5d7-24801f82c048"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 105])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bdbab4fe-05a5-497c-92d5-51ab355ee8a2, kind: Service { id: 7b4111aa-1a34-450f-8a24-b1db940782bf }, name: Name("external-dns-7b4111aa-1a34-450f-8a24-b1db940782bf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 193, 141])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 57c41de5-57cc-4ec0-b2a4-ec18275db374, kind: Service { id: 75369688-7093-4009-81fe-7452642ec8ea }, name: Name("nexus-75369688-7093-4009-81fe-7452642ec8ea"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5d808073-b6dd-4352-8286-483c26ccb36e, kind: Service { id: 80d50149-adaa-4067-b457-5cd986e26bb3 }, name: Name("nexus-80d50149-adaa-4067-b457-5cd986e26bb3"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c047ac03-3e6b-4894-a737-123382b38a57, kind: Service { id: 0cdd90ac-34d6-41d9-bd04-18f3635adaf7 }, name: Name("nexus-0cdd90ac-34d6-41d9-bd04-18f3635adaf7"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
20532024-11-28T01:40:12.725ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20542024-11-28T01:40:12.725ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20552024-11-28T01:40:12.725ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
20562024-11-28T01:40:12.725ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
20572024-11-28T01:40:12.725ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20582024-11-28T01:40:12.726ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
20592024-11-28T01:40:12.726ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 12432 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45190 req_id = 1e5b602d-6687-4238-83f2-b2a0bde452f5 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
20602024-11-28T01:40:12.762ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
20612024-11-28T01:40:12.762ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 233 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48701 req_id = 33376316-2977-40ad-affb-08f34d3e3eea response_code = 204 uri = /eip-gateways
20622024-11-28T01:40:12.762ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20632024-11-28T01:40:12.791ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44604
20642024-11-28T01:40:12.791ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 7517eed1-3882-4d66-a665-f02c99641d63, kind: Service { id: b555dc2a-491b-4b5c-b5d7-24801f82c048 }, name: Name("external-dns-b555dc2a-491b-4b5c-b5d7-24801f82c048"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 105])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bdbab4fe-05a5-497c-92d5-51ab355ee8a2, kind: Service { id: 7b4111aa-1a34-450f-8a24-b1db940782bf }, name: Name("external-dns-7b4111aa-1a34-450f-8a24-b1db940782bf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 193, 141])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 57c41de5-57cc-4ec0-b2a4-ec18275db374, kind: Service { id: 75369688-7093-4009-81fe-7452642ec8ea }, name: Name("nexus-75369688-7093-4009-81fe-7452642ec8ea"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5d808073-b6dd-4352-8286-483c26ccb36e, kind: Service { id: 80d50149-adaa-4067-b457-5cd986e26bb3 }, name: Name("nexus-80d50149-adaa-4067-b457-5cd986e26bb3"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c047ac03-3e6b-4894-a737-123382b38a57, kind: Service { id: 0cdd90ac-34d6-41d9-bd04-18f3635adaf7 }, name: Name("nexus-0cdd90ac-34d6-41d9-bd04-18f3635adaf7"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
20652024-11-28T01:40:12.791ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20662024-11-28T01:40:12.791ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20672024-11-28T01:40:12.791ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
20682024-11-28T01:40:12.791ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
20692024-11-28T01:40:12.791ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20702024-11-28T01:40:12.791ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
20712024-11-28T01:40:12.791ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 396 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44604 req_id = 9bdc12ef-d491-439c-a40e-d270afa72f1f response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
20722024-11-28T01:40:12.831ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 117855 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:53835 req_id = 6dca6e50-2175-4e75-9b2b-6c3fd220d37e response_code = 204 uri = /switch-ports
20732024-11-28T01:40:12.831ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53835 req_id = c5a44d52-a673-431c-a502-48a10e161fd6 response_code = 200 uri = /network-bootstore-config
20742024-11-28T01:40:12.882ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38625
20752024-11-28T01:40:12.882ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38625 req_id = fc2dc996-8b78-4542-9fc4-3861cb13f008 response_code = 200 uri = /v2p
20762024-11-28T01:40:12.891ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 194 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48701 req_id = d6c2b035-7c69-425d-9640-58e25a3e0d5f response_code = 204 uri = /vpc-routes
20772024-11-28T01:40:12.919ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45762
20782024-11-28T01:40:12.919ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20792024-11-28T01:40:12.961ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20802024-11-28T01:40:13.024ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104808 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:45762 req_id = b51c78fe-20c9-48a4-b322-c696cf1d65e6 response_code = 204 uri = /switch-ports
20812024-11-28T01:40:13.024ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45762 req_id = bf4bb61c-4d7e-4c87-a8c7-d0e9c8a0ab31 response_code = 200 uri = /network-bootstore-config
20822024-11-28T01:40:13.110ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35244
20832024-11-28T01:40:13.110ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 7517eed1-3882-4d66-a665-f02c99641d63, kind: Service { id: b555dc2a-491b-4b5c-b5d7-24801f82c048 }, name: Name("external-dns-b555dc2a-491b-4b5c-b5d7-24801f82c048"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 105])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bdbab4fe-05a5-497c-92d5-51ab355ee8a2, kind: Service { id: 7b4111aa-1a34-450f-8a24-b1db940782bf }, name: Name("external-dns-7b4111aa-1a34-450f-8a24-b1db940782bf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 193, 141])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 57c41de5-57cc-4ec0-b2a4-ec18275db374, kind: Service { id: 75369688-7093-4009-81fe-7452642ec8ea }, name: Name("nexus-75369688-7093-4009-81fe-7452642ec8ea"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5d808073-b6dd-4352-8286-483c26ccb36e, kind: Service { id: 80d50149-adaa-4067-b457-5cd986e26bb3 }, name: Name("nexus-80d50149-adaa-4067-b457-5cd986e26bb3"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c047ac03-3e6b-4894-a737-123382b38a57, kind: Service { id: 0cdd90ac-34d6-41d9-bd04-18f3635adaf7 }, name: Name("nexus-0cdd90ac-34d6-41d9-bd04-18f3635adaf7"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
20842024-11-28T01:40:13.110ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20852024-11-28T01:40:13.110ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20862024-11-28T01:40:13.110ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
20872024-11-28T01:40:13.110ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
20882024-11-28T01:40:13.110ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20892024-11-28T01:40:13.110ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
20902024-11-28T01:40:13.110ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 269 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35244 req_id = 0d1837a7-7c6f-4cbd-b9f3-7d9dfbd82597 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
20912024-11-28T01:40:26.952ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:54036
20922024-11-28T01:40:26.952ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 33 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:54036 req_id = 8695d90c-be49-4186-9c73-70ff95e4aebe response_code = 200 uri = /baseboard
20932024-11-28T01:40:42.202ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49906
20942024-11-28T01:40:42.202ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49906 req_id = b46720ec-ba2a-4784-849d-dd8db9e2c1db response_code = 200 uri = /v2p
20952024-11-28T01:40:42.224ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36871
20962024-11-28T01:40:42.224ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36871 req_id = 8bb57b30-efb3-4f27-a55b-faff9f7616fc response_code = 200 uri = /vpc-routes
20972024-11-28T01:40:42.245ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
20982024-11-28T01:40:42.245ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:36871 req_id = c3d948e0-0ecf-4fb4-bf7f-88e960b5c349 response_code = 204 uri = /eip-gateways
20992024-11-28T01:40:42.256ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 162 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:36871 req_id = b84bde37-a1bd-4c02-b190-d180aa0ef6f8 response_code = 204 uri = /vpc-routes
21002024-11-28T01:40:42.304ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39435
21012024-11-28T01:40:42.305ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 31 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39435 req_id = fdd22a91-eab6-4b5f-a15e-902702bd1ece response_code = 200 uri = /vpc-routes
21022024-11-28T01:40:42.333ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
21032024-11-28T01:40:42.333ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:39435 req_id = bb9dd1d5-4b0d-4ee4-955a-ec09b33b52f8 response_code = 204 uri = /eip-gateways
21042024-11-28T01:40:42.338ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37903
21052024-11-28T01:40:42.338ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21062024-11-28T01:40:42.397ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21072024-11-28T01:40:42.474ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 136489 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:37903 req_id = 4997ef2d-6a4c-4c6d-ae0e-86f0339ba089 response_code = 204 uri = /switch-ports
21082024-11-28T01:40:42.475ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53449
21092024-11-28T01:40:42.475ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58512
21102024-11-28T01:40:42.475ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 359 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37903 req_id = 573218f5-2ff4-4433-a21f-5608535be653 response_code = 200 uri = /network-bootstore-config
21112024-11-28T01:40:42.475ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 216 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53449 req_id = 6b841caa-be3d-43c6-9e97-1fc5d0470012 response_code = 200 uri = /v2p
21122024-11-28T01:40:42.475ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21132024-11-28T01:40:42.481ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6727 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:39435 req_id = f8614167-d4db-444d-aa89-ea591c0d5a44 response_code = 204 uri = /vpc-routes
21142024-11-28T01:40:42.520ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21152024-11-28T01:40:42.560ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54767
21162024-11-28T01:40:42.560ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54767 req_id = 7507a8cb-9c32-4f67-8002-21681b241584 response_code = 200 uri = /vpc-routes
21172024-11-28T01:40:42.580ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105417 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:58512 req_id = 2a11dd45-1f8d-457b-a2af-013fb6cb90df response_code = 204 uri = /switch-ports
21182024-11-28T01:40:42.581ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 138 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58512 req_id = dc842633-d439-4440-b8d2-48f03dbcd38c response_code = 200 uri = /network-bootstore-config
21192024-11-28T01:40:42.604ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
21202024-11-28T01:40:42.604ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 81 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54767 req_id = 68e62e81-e813-4228-b3a6-bc12692bc946 response_code = 204 uri = /eip-gateways
21212024-11-28T01:40:42.613ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 242 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54767 req_id = 98c1e6c4-3808-45d7-b5bc-5193c63856f2 response_code = 204 uri = /vpc-routes
21222024-11-28T01:40:42.643ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56928
21232024-11-28T01:40:42.643ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21242024-11-28T01:40:42.685ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21252024-11-28T01:40:42.748ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104526 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:56928 req_id = c7a04247-21d9-4147-90ec-459f2c659568 response_code = 204 uri = /switch-ports
21262024-11-28T01:40:42.748ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34339
21272024-11-28T01:40:42.748ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34339 req_id = ac583705-12c4-4748-80c5-6cd1dd19c410 response_code = 200 uri = /v2p
21282024-11-28T01:40:42.748ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56928 req_id = cff0a330-f9f6-476d-9cce-3da3c9db5d67 response_code = 200 uri = /network-bootstore-config
21292024-11-28T01:40:56.985ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43039
21302024-11-28T01:40:56.986ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 26 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43039 req_id = 49357e18-4510-459d-9f12-b777383cb691 response_code = 200 uri = /baseboard
21312024-11-28T01:41:06.493ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21322024-11-28T01:41:06.495ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21332024-11-28T01:41:06.495ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21342024-11-28T01:41:06.496ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21352024-11-28T01:41:06.497ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21362024-11-28T01:41:06.497ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21372024-11-28T01:41:06.498ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21382024-11-28T01:41:06.499ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21392024-11-28T01:41:06.499ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21402024-11-28T01:41:06.500ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21412024-11-28T01:41:06.501ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21422024-11-28T01:41:06.501ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21432024-11-28T01:41:06.502ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21442024-11-28T01:41:06.505ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21452024-11-28T01:41:06.505ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21462024-11-28T01:41:06.506ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21472024-11-28T01:41:06.507ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21482024-11-28T01:41:06.508ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21492024-11-28T01:41:12.315ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63072
21502024-11-28T01:41:12.315ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 73 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63072 req_id = ed82ef54-56d8-43c5-aeeb-d7c10e4246a6 response_code = 200 uri = /vpc-routes
21512024-11-28T01:41:12.363ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42153
21522024-11-28T01:41:12.363ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42153 req_id = df3682a2-7396-48dd-9123-024759d62bee response_code = 200 uri = /vpc-routes
21532024-11-28T01:41:12.426ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
21542024-11-28T01:41:12.426ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63072 req_id = 2119b482-de02-42f4-b447-99c01eade971 response_code = 204 uri = /eip-gateways
21552024-11-28T01:41:12.439ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 255 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63072 req_id = 575fe30f-eed8-41b3-bdb0-a73b47cdce00 response_code = 204 uri = /vpc-routes
21562024-11-28T01:41:12.462ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
21572024-11-28T01:41:12.462ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42153 req_id = 2355ecd0-57d8-4f87-9f43-61d1e292e93d response_code = 204 uri = /eip-gateways
21582024-11-28T01:41:12.467ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43898
21592024-11-28T01:41:12.468ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 201 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43898 req_id = 065ece36-025a-4e34-b61a-4735ebe899cc response_code = 200 uri = /v2p
21602024-11-28T01:41:12.482ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 268 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42153 req_id = 7c86f5a2-f5cf-466c-88cc-37c318355066 response_code = 204 uri = /vpc-routes
21612024-11-28T01:41:12.558ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37985
21622024-11-28T01:41:12.558ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21632024-11-28T01:41:12.600ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21642024-11-28T01:41:12.663ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105380 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:37985 req_id = 12763387-cf26-4b1a-8f97-ad6d866e050a response_code = 204 uri = /switch-ports
21652024-11-28T01:41:12.663ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41354
21662024-11-28T01:41:12.663ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41354 req_id = d825ad1e-8ab0-434f-b311-64110fc03a82 response_code = 200 uri = /v2p
21672024-11-28T01:41:12.663ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37985 req_id = 130de739-4c4b-47b7-8575-fd1b3a202edd response_code = 200 uri = /network-bootstore-config
21682024-11-28T01:41:12.806ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51323
21692024-11-28T01:41:12.806ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21702024-11-28T01:41:12.847ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21712024-11-28T01:41:12.910ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104006 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:51323 req_id = da27dfc0-e8bd-461b-a4ee-97c6e91f5a62 response_code = 204 uri = /switch-ports
21722024-11-28T01:41:12.910ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64718
21732024-11-28T01:41:12.910ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64718 req_id = 47532dfa-9d53-4ff9-8dcf-2ea4f7672f8f response_code = 200 uri = /vpc-routes
21742024-11-28T01:41:12.910ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51323 req_id = cb37f128-cf1b-4c98-ac8c-83966dc26e05 response_code = 200 uri = /network-bootstore-config
21752024-11-28T01:41:12.926ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
21762024-11-28T01:41:12.926ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:64718 req_id = 8e60f841-7fc5-4101-b707-3de355967ad9 response_code = 204 uri = /eip-gateways
21772024-11-28T01:41:12.936ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 241 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:64718 req_id = bc74b22f-b060-4442-9879-aa8a3fa9cfa3 response_code = 204 uri = /vpc-routes
21782024-11-28T01:41:12.945ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56170
21792024-11-28T01:41:12.945ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21802024-11-28T01:41:12.993ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21812024-11-28T01:41:13.057ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 112050 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:56170 req_id = 0dfd36f9-7c07-476d-ac33-1ccfc1e2d44b response_code = 204 uri = /switch-ports
21822024-11-28T01:41:13.057ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45013
21832024-11-28T01:41:13.057ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 179 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45013 req_id = a4847d0b-0f20-4180-b0d9-8ffb230787e2 response_code = 200 uri = /v2p
21842024-11-28T01:41:13.057ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 247 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56170 req_id = 6b560e43-05ff-4e32-a496-8cac4fcc21d5 response_code = 200 uri = /network-bootstore-config
21852024-11-28T01:41:16.499ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21862024-11-28T01:41:16.500ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21872024-11-28T01:41:16.500ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21882024-11-28T01:41:16.501ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21892024-11-28T01:41:16.502ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21902024-11-28T01:41:16.502ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21912024-11-28T01:41:16.503ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21922024-11-28T01:41:16.504ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21932024-11-28T01:41:16.504ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21942024-11-28T01:41:16.505ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21952024-11-28T01:41:16.506ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21962024-11-28T01:41:16.506ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21972024-11-28T01:41:16.507ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21982024-11-28T01:41:16.509ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
21992024-11-28T01:41:16.509ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22002024-11-28T01:41:16.510ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22012024-11-28T01:41:16.511ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22022024-11-28T01:41:16.512ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22032024-11-28T01:41:26.503ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22042024-11-28T01:41:26.503ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22052024-11-28T01:41:26.504ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22062024-11-28T01:41:26.505ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22072024-11-28T01:41:26.505ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22082024-11-28T01:41:26.506ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22092024-11-28T01:41:26.507ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22102024-11-28T01:41:26.507ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22112024-11-28T01:41:26.508ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22122024-11-28T01:41:26.509ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22132024-11-28T01:41:26.509ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22142024-11-28T01:41:26.510ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22152024-11-28T01:41:26.511ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22162024-11-28T01:41:26.512ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22172024-11-28T01:41:26.513ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22182024-11-28T01:41:26.513ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22192024-11-28T01:41:26.515ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22202024-11-28T01:41:26.516ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22212024-11-28T01:41:27.021ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:54800
22222024-11-28T01:41:27.021ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 31 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:54800 req_id = 6aa006d6-bb06-45ee-a745-b74d313b2857 response_code = 200 uri = /baseboard
22232024-11-28T01:41:36.506ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22242024-11-28T01:41:36.507ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22252024-11-28T01:41:36.508ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22262024-11-28T01:41:36.509ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22272024-11-28T01:41:36.509ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22282024-11-28T01:41:36.510ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22292024-11-28T01:41:36.510ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22302024-11-28T01:41:36.511ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22312024-11-28T01:41:36.512ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22322024-11-28T01:41:36.512ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22332024-11-28T01:41:36.513ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22342024-11-28T01:41:36.514ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22352024-11-28T01:41:36.515ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22362024-11-28T01:41:36.516ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22372024-11-28T01:41:36.517ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22382024-11-28T01:41:36.517ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22392024-11-28T01:41:36.519ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22402024-11-28T01:41:36.520ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22412024-11-28T01:41:42.203ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50948
22422024-11-28T01:41:42.203ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50948 req_id = f0a8fa12-69b7-47b9-bbf6-49a390247740 response_code = 200 uri = /vpc-routes
22432024-11-28T01:41:42.243ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56507
22442024-11-28T01:41:42.243ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 149 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56507 req_id = 4a1b347b-84fe-4954-88b9-8bc33bec49f6 response_code = 200 uri = /v2p
22452024-11-28T01:41:42.253ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
22462024-11-28T01:41:42.253ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:50948 req_id = 886538b6-9b15-4ea7-9501-22c19a3147fa response_code = 204 uri = /eip-gateways
22472024-11-28T01:41:42.263ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 172 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:50948 req_id = e0a25d90-2fcd-4dd1-ad34-39e35862e402 response_code = 204 uri = /vpc-routes
22482024-11-28T01:41:42.317ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42200
22492024-11-28T01:41:42.317ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
22502024-11-28T01:41:42.376ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
22512024-11-28T01:41:42.453ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 136779 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:42200 req_id = c99421a3-d787-4ea7-837b-c849e6a38af4 response_code = 204 uri = /switch-ports
22522024-11-28T01:41:42.454ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42751
22532024-11-28T01:41:42.454ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45015
22542024-11-28T01:41:42.454ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64284
22552024-11-28T01:41:42.454ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 140 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42751 req_id = 2ed4a912-08cd-48d3-9a25-29e24c87c02d response_code = 200 uri = /vpc-routes
22562024-11-28T01:41:42.454ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 259 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42200 req_id = bbdb6a39-8c6c-4731-bf02-0d89bf4dcb6a response_code = 200 uri = /network-bootstore-config
22572024-11-28T01:41:42.454ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
22582024-11-28T01:41:42.454ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 328 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64284 req_id = 4e1961e2-0939-4024-971b-f15a16e7fe83 response_code = 200 uri = /v2p
22592024-11-28T01:41:42.497ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
22602024-11-28T01:41:42.497ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42751 req_id = 01b1ee80-d880-4e91-8bf6-8c6367bcec89 response_code = 204 uri = /eip-gateways
22612024-11-28T01:41:42.499ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
22622024-11-28T01:41:42.511ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 294 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42751 req_id = e2ee300a-e63d-4bb1-ba7b-dc2f28645089 response_code = 204 uri = /vpc-routes
22632024-11-28T01:41:42.566ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111839 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:45015 req_id = 2c764571-28ed-44e5-91b0-aa09a889b1ba response_code = 204 uri = /switch-ports
22642024-11-28T01:41:42.566ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45015 req_id = 074ab039-e1a9-485b-9b30-419519a85780 response_code = 200 uri = /network-bootstore-config
22652024-11-28T01:41:42.568ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33512
22662024-11-28T01:41:42.568ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33512 req_id = 26ea0a12-d52b-4e2f-9189-d30bbdc048b2 response_code = 200 uri = /vpc-routes
22672024-11-28T01:41:42.608ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52904
22682024-11-28T01:41:42.608ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52904 req_id = e2883287-22f3-4ece-95ae-543574be262a response_code = 200 uri = /v2p
22692024-11-28T01:41:42.615ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
22702024-11-28T01:41:42.615ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:33512 req_id = 3ef446c7-3378-475b-bdcb-722887b15ce0 response_code = 204 uri = /eip-gateways
22712024-11-28T01:41:42.624ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 206 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:33512 req_id = 9efb6f4e-3fc8-4c3b-9d32-a938641c7067 response_code = 204 uri = /vpc-routes
22722024-11-28T01:41:42.677ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54271
22732024-11-28T01:41:42.677ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
22742024-11-28T01:41:42.717ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
22752024-11-28T01:41:42.779ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101380 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:54271 req_id = ee57e92b-3b36-46d6-935f-38df9fb0f8ee response_code = 204 uri = /switch-ports
22762024-11-28T01:41:42.779ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54271 req_id = 4eadf758-c34f-4780-88df-0a129a0bac48 response_code = 200 uri = /network-bootstore-config
22772024-11-28T01:41:46.511ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22782024-11-28T01:41:46.512ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22792024-11-28T01:41:46.512ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22802024-11-28T01:41:46.513ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22812024-11-28T01:41:46.513ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22822024-11-28T01:41:46.514ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22832024-11-28T01:41:46.515ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22842024-11-28T01:41:46.515ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22852024-11-28T01:41:46.516ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22862024-11-28T01:41:46.517ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22872024-11-28T01:41:46.517ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22882024-11-28T01:41:46.518ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22892024-11-28T01:41:46.529ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22902024-11-28T01:41:46.530ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22912024-11-28T01:41:46.531ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22922024-11-28T01:41:46.531ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22932024-11-28T01:41:46.532ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22942024-11-28T01:41:46.533ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22952024-11-28T01:41:56.515ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22962024-11-28T01:41:56.516ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22972024-11-28T01:41:56.517ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22982024-11-28T01:41:56.517ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
22992024-11-28T01:41:56.518ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23002024-11-28T01:41:56.518ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23012024-11-28T01:41:56.519ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23022024-11-28T01:41:56.520ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23032024-11-28T01:41:56.520ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23042024-11-28T01:41:56.521ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23052024-11-28T01:41:56.522ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23062024-11-28T01:41:56.531ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23072024-11-28T01:41:56.531ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23082024-11-28T01:41:56.533ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23092024-11-28T01:41:56.533ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23102024-11-28T01:41:56.534ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23112024-11-28T01:41:56.535ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23122024-11-28T01:41:56.536ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23132024-11-28T01:41:57.055ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61972
23142024-11-28T01:41:57.055ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 32 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:61972 req_id = 79286c77-9d1a-4a32-8258-732944e48988 response_code = 200 uri = /baseboard
23152024-11-28T01:42:06.519ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23162024-11-28T01:42:06.520ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23172024-11-28T01:42:06.521ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23182024-11-28T01:42:06.522ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23192024-11-28T01:42:06.522ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23202024-11-28T01:42:06.523ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23212024-11-28T01:42:06.524ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23222024-11-28T01:42:06.524ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23232024-11-28T01:42:06.525ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23242024-11-28T01:42:06.525ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23252024-11-28T01:42:06.526ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23262024-11-28T01:42:06.532ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23272024-11-28T01:42:06.532ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23282024-11-28T01:42:06.534ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23292024-11-28T01:42:06.534ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23302024-11-28T01:42:06.535ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23312024-11-28T01:42:06.536ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23322024-11-28T01:42:06.537ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23332024-11-28T01:42:12.376ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34849
23342024-11-28T01:42:12.376ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34849 req_id = 84536c4c-1d2f-473b-8a33-03e0d19903a5 response_code = 200 uri = /vpc-routes
23352024-11-28T01:42:12.401ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
23362024-11-28T01:42:12.401ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34849 req_id = d3e2d771-7551-41d9-a660-a5db1ea257ce response_code = 204 uri = /eip-gateways
23372024-11-28T01:42:12.409ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62685
23382024-11-28T01:42:12.409ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62685 req_id = 4dd54675-b5ad-4e49-92be-44fa61964958 response_code = 200 uri = /vpc-routes
23392024-11-28T01:42:12.416ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 242 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34849 req_id = d95fbd69-3866-443a-9c12-61a31cc04c63 response_code = 204 uri = /vpc-routes
23402024-11-28T01:42:12.433ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
23412024-11-28T01:42:12.433ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:62685 req_id = d82c87c7-6991-46f0-a970-cc95d61133d9 response_code = 204 uri = /eip-gateways
23422024-11-28T01:42:12.443ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 200 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:62685 req_id = bac520d2-f16a-42f6-a088-7cb371b99f35 response_code = 204 uri = /vpc-routes
23432024-11-28T01:42:12.501ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38541
23442024-11-28T01:42:12.501ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 144 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38541 req_id = 578421ec-40fb-4966-ab05-6ef808dc7e7d response_code = 200 uri = /v2p
23452024-11-28T01:42:12.700ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43219
23462024-11-28T01:42:12.700ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43219 req_id = cfabdcbc-0dd7-45e3-bc3e-c9e1e66012ea response_code = 200 uri = /v2p
23472024-11-28T01:42:12.716ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34632
23482024-11-28T01:42:12.716ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
23492024-11-28T01:42:12.755ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
23502024-11-28T01:42:12.820ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104629 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:34632 req_id = d68e327c-89e0-4a48-a20b-187573a07ec3 response_code = 204 uri = /switch-ports
23512024-11-28T01:42:12.821ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63166
23522024-11-28T01:42:12.821ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42987
23532024-11-28T01:42:12.821ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41569
23542024-11-28T01:42:12.825ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
23552024-11-28T01:42:12.825ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4588 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34632 req_id = 0c78d6dc-5cb2-429a-9120-8308f81a8e18 response_code = 200 uri = /network-bootstore-config
23562024-11-28T01:42:12.826ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4554 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63166 req_id = 823912b6-7928-4a5c-80dc-a89e8b48378b response_code = 200 uri = /vpc-routes
23572024-11-28T01:42:12.832ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 10979 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42987 req_id = 8346a675-72d4-4353-967b-038c423207f3 response_code = 200 uri = /v2p
23582024-11-28T01:42:12.873ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
23592024-11-28T01:42:12.879ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
23602024-11-28T01:42:12.879ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 667 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63166 req_id = 843f2f97-723a-483e-8e45-b353e1166b6c response_code = 204 uri = /eip-gateways
23612024-11-28T01:42:12.893ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4700 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63166 req_id = b15afd43-3b6a-41cf-bd88-67980e4ee7c2 response_code = 204 uri = /vpc-routes
23622024-11-28T01:42:12.933ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43573
23632024-11-28T01:42:12.940ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
23642024-11-28T01:42:12.940ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 115418 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:41569 req_id = c22e4db3-1704-4d2a-bade-c770a074cc19 response_code = 204 uri = /switch-ports
23652024-11-28T01:42:12.946ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 773 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41569 req_id = a0b352bc-9c47-41fa-acfd-3b31ab21f7fe response_code = 200 uri = /network-bootstore-config
23662024-11-28T01:42:12.981ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
23672024-11-28T01:42:13.034ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100931 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:43573 req_id = 6a5c408e-930c-43f3-a11f-4d2c9d1e0f82 response_code = 204 uri = /switch-ports
23682024-11-28T01:42:13.034ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 159 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43573 req_id = d48ddb9d-87de-46ec-92cf-bc936cfa38df response_code = 200 uri = /network-bootstore-config
23692024-11-28T01:42:16.525ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23702024-11-28T01:42:16.525ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23712024-11-28T01:42:16.526ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23722024-11-28T01:42:16.527ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23732024-11-28T01:42:16.527ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23742024-11-28T01:42:16.528ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23752024-11-28T01:42:16.529ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23762024-11-28T01:42:16.529ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23772024-11-28T01:42:16.530ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23782024-11-28T01:42:16.530ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23792024-11-28T01:42:16.531ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23802024-11-28T01:42:16.534ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23812024-11-28T01:42:16.535ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23822024-11-28T01:42:16.536ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23832024-11-28T01:42:16.537ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23842024-11-28T01:42:16.537ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23852024-11-28T01:42:16.539ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23862024-11-28T01:42:16.540ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23872024-11-28T01:42:26.528ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23882024-11-28T01:42:26.529ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23892024-11-28T01:42:26.530ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23902024-11-28T01:42:26.531ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23912024-11-28T01:42:26.531ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23922024-11-28T01:42:26.532ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23932024-11-28T01:42:26.532ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23942024-11-28T01:42:26.533ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23952024-11-28T01:42:26.534ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23962024-11-28T01:42:26.534ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23972024-11-28T01:42:26.535ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23982024-11-28T01:42:26.537ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
23992024-11-28T01:42:26.537ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24002024-11-28T01:42:26.539ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24012024-11-28T01:42:26.539ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24022024-11-28T01:42:26.540ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24032024-11-28T01:42:26.541ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24042024-11-28T01:42:26.542ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24052024-11-28T01:42:27.089ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:57018
24062024-11-28T01:42:27.089ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 39 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:57018 req_id = 665f572a-8bc1-46e4-b0c8-1178aed30283 response_code = 200 uri = /baseboard
24072024-11-28T01:42:36.533ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24082024-11-28T01:42:36.534ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24092024-11-28T01:42:36.534ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24102024-11-28T01:42:36.535ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24112024-11-28T01:42:36.536ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24122024-11-28T01:42:36.536ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24132024-11-28T01:42:36.537ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24142024-11-28T01:42:36.537ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24152024-11-28T01:42:36.538ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24162024-11-28T01:42:36.539ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24172024-11-28T01:42:36.539ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24182024-11-28T01:42:36.540ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24192024-11-28T01:42:36.541ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24202024-11-28T01:42:36.542ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24212024-11-28T01:42:36.542ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24222024-11-28T01:42:36.543ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24232024-11-28T01:42:36.544ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24242024-11-28T01:42:36.545ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24252024-11-28T01:42:42.131ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45973
24262024-11-28T01:42:42.131ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45973 req_id = 3208af27-bd18-4c3e-9726-3df011725a5d response_code = 200 uri = /vpc-routes
24272024-11-28T01:42:42.154ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
24282024-11-28T01:42:42.154ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45973 req_id = cdaf55c4-1c28-43b1-b834-dc9a2e38bd2f response_code = 204 uri = /eip-gateways
24292024-11-28T01:42:42.171ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 204 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45973 req_id = f35cbaaf-c48f-4298-802f-6ba465fc4e07 response_code = 204 uri = /vpc-routes
24302024-11-28T01:42:42.292ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44546
24312024-11-28T01:42:42.292ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44546 req_id = be2124c5-917b-4bbc-acd1-e8b66d26cb78 response_code = 200 uri = /v2p
24322024-11-28T01:42:42.298ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64836
24332024-11-28T01:42:42.298ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 34 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64836 req_id = 0a7fa6be-230c-4823-8320-cc95cf64dd82 response_code = 200 uri = /vpc-routes
24342024-11-28T01:42:42.367ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
24352024-11-28T01:42:42.367ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64836 req_id = 22b2ee1e-0e68-47aa-a87d-4b389a4c7cb3 response_code = 204 uri = /eip-gateways
24362024-11-28T01:42:42.378ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 207 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64836 req_id = c80bdf51-91f7-469c-be86-cbfe4393c959 response_code = 204 uri = /vpc-routes
24372024-11-28T01:42:42.387ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53150
24382024-11-28T01:42:42.387ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
24392024-11-28T01:42:42.434ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
24402024-11-28T01:42:42.502ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 115456 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:53150 req_id = dca5fef0-746b-47c2-a87e-6a32d59119ac response_code = 204 uri = /switch-ports
24412024-11-28T01:42:42.503ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45239
24422024-11-28T01:42:42.503ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35801
24432024-11-28T01:42:42.503ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43602
24442024-11-28T01:42:42.503ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 292 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53150 req_id = bf02baf7-2c2b-43ed-be49-7b6e40f26c7d response_code = 200 uri = /network-bootstore-config
24452024-11-28T01:42:42.503ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43602 req_id = ad186c65-317a-4a0f-a689-121597731c5e response_code = 200 uri = /vpc-routes
24462024-11-28T01:42:42.503ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
24472024-11-28T01:42:42.503ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 234 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35801 req_id = 1fed99e8-e1e9-4d57-ad4a-87ef77da937d response_code = 200 uri = /v2p
24482024-11-28T01:42:42.548ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
24492024-11-28T01:42:42.548ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 140 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:43602 req_id = 19c94f4f-636b-410f-bd41-d0a2dccb2239 response_code = 204 uri = /eip-gateways
24502024-11-28T01:42:42.549ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
24512024-11-28T01:42:42.557ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 375 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:43602 req_id = ed7c3968-7c41-4cbe-a147-a353edca187c response_code = 204 uri = /vpc-routes
24522024-11-28T01:42:42.607ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33914
24532024-11-28T01:42:42.608ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 222 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33914 req_id = 73e2c142-934f-4200-b033-96fd4ee437b9 response_code = 200 uri = /v2p
24542024-11-28T01:42:42.617ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113719 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:45239 req_id = 7fd49682-55bd-42d8-b585-1552a12ba093 response_code = 204 uri = /switch-ports
24552024-11-28T01:42:42.617ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45239 req_id = 22f76a06-d692-49a9-b6cb-975716ba03c7 response_code = 200 uri = /network-bootstore-config
24562024-11-28T01:42:42.649ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40671
24572024-11-28T01:42:42.649ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
24582024-11-28T01:42:42.687ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
24592024-11-28T01:42:42.748ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99281 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:40671 req_id = 6a5c17e6-b6f4-4c15-ab7b-30d4a629ce26 response_code = 204 uri = /switch-ports
24602024-11-28T01:42:42.748ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40671 req_id = 3be4f201-da11-4b3a-903f-f62c7ebf80ba response_code = 200 uri = /network-bootstore-config
24612024-11-28T01:42:46.537ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24622024-11-28T01:42:46.537ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24632024-11-28T01:42:46.538ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24642024-11-28T01:42:46.539ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24652024-11-28T01:42:46.539ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24662024-11-28T01:42:46.540ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24672024-11-28T01:42:46.540ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24682024-11-28T01:42:46.541ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24692024-11-28T01:42:46.542ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24702024-11-28T01:42:46.542ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24712024-11-28T01:42:46.543ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24722024-11-28T01:42:46.544ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24732024-11-28T01:42:46.544ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24742024-11-28T01:42:46.546ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24752024-11-28T01:42:46.546ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24762024-11-28T01:42:46.547ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24772024-11-28T01:42:46.548ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24782024-11-28T01:42:46.549ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24792024-11-28T01:42:56.540ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24802024-11-28T01:42:56.540ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24812024-11-28T01:42:56.541ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24822024-11-28T01:42:56.542ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24832024-11-28T01:42:56.542ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24842024-11-28T01:42:56.543ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24852024-11-28T01:42:56.544ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24862024-11-28T01:42:56.544ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24872024-11-28T01:42:56.545ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24882024-11-28T01:42:56.545ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24892024-11-28T01:42:56.546ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24902024-11-28T01:42:56.547ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24912024-11-28T01:42:56.547ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24922024-11-28T01:42:56.549ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24932024-11-28T01:42:56.549ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24942024-11-28T01:42:56.550ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24952024-11-28T01:42:56.551ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24962024-11-28T01:42:56.552ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
24972024-11-28T01:42:57.121ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:45822
24982024-11-28T01:42:57.121ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:45822 req_id = 69ed663c-4e0f-4693-a661-63520e77f244 response_code = 200 uri = /baseboard
24992024-11-28T01:43:06.013ZINFOSledAgent (auto-cleanup-task): running automatic periodic zone bundle cleanup file = sled-agent/src/zone_bundle.rs:197
25002024-11-28T01:43:06.014ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
25012024-11-28T01:43:06.056ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 3011, tv_nsec: 374820033 } time_to_next_cleanup = 599.99999165s
25022024-11-28T01:43:06.544ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25032024-11-28T01:43:06.544ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25042024-11-28T01:43:06.545ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25052024-11-28T01:43:06.546ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25062024-11-28T01:43:06.546ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25072024-11-28T01:43:06.547ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25082024-11-28T01:43:06.547ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25092024-11-28T01:43:06.548ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25102024-11-28T01:43:06.549ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25112024-11-28T01:43:06.549ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25122024-11-28T01:43:06.550ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25132024-11-28T01:43:06.551ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25142024-11-28T01:43:06.551ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25152024-11-28T01:43:06.553ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25162024-11-28T01:43:06.553ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25172024-11-28T01:43:06.554ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25182024-11-28T01:43:06.555ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25192024-11-28T01:43:06.556ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25202024-11-28T01:43:12.350ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53449
25212024-11-28T01:43:12.350ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 178 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53449 req_id = 3082967e-3fb6-4ede-a90b-fb97bae39c44 response_code = 200 uri = /v2p
25222024-11-28T01:43:12.378ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56837
25232024-11-28T01:43:12.378ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56837 req_id = f155ee10-15f1-49e5-8114-785169891134 response_code = 200 uri = /v2p
25242024-11-28T01:43:12.412ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38544
25252024-11-28T01:43:12.412ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38544 req_id = 1fc034e8-acee-40e9-b6cc-640baee84d94 response_code = 200 uri = /vpc-routes
25262024-11-28T01:43:12.504ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
25272024-11-28T01:43:12.504ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:38544 req_id = c8d4cf39-4175-4cc2-8ee6-bcc3abb82609 response_code = 204 uri = /eip-gateways
25282024-11-28T01:43:12.519ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 254 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:38544 req_id = d2caa8b0-bf21-473b-bcb9-c05bee9a519f response_code = 204 uri = /vpc-routes
25292024-11-28T01:43:12.544ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40532
25302024-11-28T01:43:12.544ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
25312024-11-28T01:43:12.590ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
25322024-11-28T01:43:12.660ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 115772 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:40532 req_id = 0cd70843-54d6-434e-916f-5333c0e1ff04 response_code = 204 uri = /switch-ports
25332024-11-28T01:43:12.660ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61159
25342024-11-28T01:43:12.660ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55758
25352024-11-28T01:43:12.660ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 170 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40532 req_id = 4cee45ec-3cd3-4cae-9332-93c04287425f response_code = 200 uri = /network-bootstore-config
25362024-11-28T01:43:12.660ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61159 req_id = ade598ee-4d2f-49fc-be1c-8829dd86ae3f response_code = 200 uri = /vpc-routes
25372024-11-28T01:43:12.660ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
25382024-11-28T01:43:12.667ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43805
25392024-11-28T01:43:12.667ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 164 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43805 req_id = 207ba0d8-0d83-4af1-9e6a-0e9f846af73f response_code = 200 uri = /vpc-routes
25402024-11-28T01:43:12.705ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
25412024-11-28T01:43:12.711ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
25422024-11-28T01:43:12.711ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 688 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61159 req_id = 9be9f4cb-655c-4367-a854-ffb840054c1a response_code = 204 uri = /eip-gateways
25432024-11-28T01:43:12.726ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 770 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61159 req_id = 061f6d2e-4637-44b8-83e4-b3b6064b3787 response_code = 204 uri = /vpc-routes
25442024-11-28T01:43:12.732ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33692
25452024-11-28T01:43:12.732ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33692 req_id = bad4778e-71dd-493f-9c27-5f3e40834a69 response_code = 200 uri = /v2p
25462024-11-28T01:43:12.779ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 118756 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:55758 req_id = 64bc81a3-d5e6-4541-9f88-708077e3db72 response_code = 204 uri = /switch-ports
25472024-11-28T01:43:12.779ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 152 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55758 req_id = 971cf779-df7b-4d74-ac87-fb5eb496e57a response_code = 200 uri = /network-bootstore-config
25482024-11-28T01:43:12.783ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
25492024-11-28T01:43:12.783ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 75 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:43805 req_id = 94002372-c9f9-4076-b7be-20cb85c1a970 response_code = 204 uri = /eip-gateways
25502024-11-28T01:43:12.792ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51125
25512024-11-28T01:43:12.792ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
25522024-11-28T01:43:12.835ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
25532024-11-28T01:43:12.895ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103147 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:51125 req_id = 3a9faac3-9d1a-415d-9807-faf6517e24c4 response_code = 204 uri = /switch-ports
25542024-11-28T01:43:12.896ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 263 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:43805 req_id = f3564336-ac03-4940-a711-e147bcbc3c8c response_code = 204 uri = /vpc-routes
25552024-11-28T01:43:12.896ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 186 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51125 req_id = b4b1392e-054b-412f-bc50-b381ed057d4d response_code = 200 uri = /network-bootstore-config
25562024-11-28T01:43:16.548ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25572024-11-28T01:43:16.549ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25582024-11-28T01:43:16.549ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25592024-11-28T01:43:16.550ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25602024-11-28T01:43:16.550ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25612024-11-28T01:43:16.551ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25622024-11-28T01:43:16.551ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25632024-11-28T01:43:16.552ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25642024-11-28T01:43:16.553ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25652024-11-28T01:43:16.553ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25662024-11-28T01:43:16.554ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25672024-11-28T01:43:16.554ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25682024-11-28T01:43:16.555ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25692024-11-28T01:43:16.556ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25702024-11-28T01:43:16.557ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25712024-11-28T01:43:16.557ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25722024-11-28T01:43:16.559ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25732024-11-28T01:43:16.559ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25742024-11-28T01:43:20.961ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
25752024-11-28T01:43:26.552ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25762024-11-28T01:43:26.553ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25772024-11-28T01:43:26.554ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25782024-11-28T01:43:26.554ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25792024-11-28T01:43:26.555ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25802024-11-28T01:43:26.555ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25812024-11-28T01:43:26.556ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25822024-11-28T01:43:26.557ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25832024-11-28T01:43:26.557ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25842024-11-28T01:43:26.558ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25852024-11-28T01:43:26.559ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25862024-11-28T01:43:26.559ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25872024-11-28T01:43:26.560ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25882024-11-28T01:43:26.561ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25892024-11-28T01:43:26.562ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25902024-11-28T01:43:26.562ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25912024-11-28T01:43:26.563ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25922024-11-28T01:43:26.564ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25932024-11-28T01:43:27.153ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:59387
25942024-11-28T01:43:27.153ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:59387 req_id = 8d6e50e3-c469-429a-bb99-5530c73a181c response_code = 200 uri = /baseboard
25952024-11-28T01:43:36.557ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25962024-11-28T01:43:36.558ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25972024-11-28T01:43:36.558ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25982024-11-28T01:43:36.559ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
25992024-11-28T01:43:36.560ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26002024-11-28T01:43:36.560ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26012024-11-28T01:43:36.561ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26022024-11-28T01:43:36.561ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26032024-11-28T01:43:36.562ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26042024-11-28T01:43:36.563ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26052024-11-28T01:43:36.563ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26062024-11-28T01:43:36.564ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26072024-11-28T01:43:36.565ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26082024-11-28T01:43:36.566ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26092024-11-28T01:43:36.567ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26102024-11-28T01:43:36.567ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26112024-11-28T01:43:36.568ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26122024-11-28T01:43:36.569ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26132024-11-28T01:43:42.243ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46268
26142024-11-28T01:43:42.243ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46268 req_id = f32d7b7a-98cd-4008-9560-2b632bfc9f17 response_code = 200 uri = /vpc-routes
26152024-11-28T01:43:42.252ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35764
26162024-11-28T01:43:42.252ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 25 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35764 req_id = bc3f5647-d94a-4250-9c0f-df4d2aac5726 response_code = 200 uri = /vpc-routes
26172024-11-28T01:43:42.271ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
26182024-11-28T01:43:42.272ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:35764 req_id = 8bd9be6c-33c4-4a9d-8ba2-506803530ba0 response_code = 204 uri = /eip-gateways
26192024-11-28T01:43:42.280ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 200 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:35764 req_id = 47d14c03-2926-4022-91ae-22388c3a0292 response_code = 204 uri = /vpc-routes
26202024-11-28T01:43:42.289ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
26212024-11-28T01:43:42.289ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 30 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:46268 req_id = 307c6dce-e4b0-49a9-bab7-b9a660feca86 response_code = 204 uri = /eip-gateways
26222024-11-28T01:43:42.296ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40271
26232024-11-28T01:43:42.296ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40271 req_id = d135b223-70c3-4f6b-8203-1ea1c375e797 response_code = 200 uri = /v2p
26242024-11-28T01:43:42.297ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:46268 req_id = 0ba7e3f7-7f53-4bbd-b9d6-0d0fba698c24 response_code = 204 uri = /vpc-routes
26252024-11-28T01:43:42.356ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41299
26262024-11-28T01:43:42.357ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
26272024-11-28T01:43:42.410ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
26282024-11-28T01:43:42.488ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 131540 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:41299 req_id = 227660d1-9e94-43db-b415-83aead26e6d3 response_code = 204 uri = /switch-ports
26292024-11-28T01:43:42.488ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57934
26302024-11-28T01:43:42.488ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
26312024-11-28T01:43:42.493ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4654 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41299 req_id = 8778ba1a-e8c7-472c-99fa-a599c310a29c response_code = 200 uri = /network-bootstore-config
26322024-11-28T01:43:42.505ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42374
26332024-11-28T01:43:42.506ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34500
26342024-11-28T01:43:42.506ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 210 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34500 req_id = b7798311-d22c-4a71-b48c-a7ba6bdfd1bf response_code = 200 uri = /vpc-routes
26352024-11-28T01:43:42.506ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 325 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42374 req_id = 7a4ae5dc-5323-4289-9ec5-6ce65946d0ba response_code = 200 uri = /v2p
26362024-11-28T01:43:42.527ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
26372024-11-28T01:43:42.549ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
26382024-11-28T01:43:42.549ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34500 req_id = 0902fb41-c07e-4a47-83e7-71cf545d462b response_code = 204 uri = /eip-gateways
26392024-11-28T01:43:42.553ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58023
26402024-11-28T01:43:42.553ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58023 req_id = 1e71d668-3775-44ff-a11d-5c1ef6362cb8 response_code = 200 uri = /v2p
26412024-11-28T01:43:42.562ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 284 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34500 req_id = 8613c9e4-5ebe-462e-9614-926f14b6b842 response_code = 204 uri = /vpc-routes
26422024-11-28T01:43:42.590ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101934 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:57934 req_id = 7eeeaf92-a888-4bf2-b74d-95606c5b35af response_code = 204 uri = /switch-ports
26432024-11-28T01:43:42.591ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 139 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57934 req_id = 1d4f8672-fe30-4053-8d33-404b0bbda407 response_code = 200 uri = /network-bootstore-config
26442024-11-28T01:43:42.601ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62225
26452024-11-28T01:43:42.602ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
26462024-11-28T01:43:42.643ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
26472024-11-28T01:43:42.699ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97926 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:62225 req_id = e619c436-70c4-4a1b-a7da-9f5dbd85941f response_code = 204 uri = /switch-ports
26482024-11-28T01:43:42.700ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62225 req_id = 77ab00d2-8c77-42ee-bfbc-93ae60477b25 response_code = 200 uri = /network-bootstore-config
26492024-11-28T01:43:46.561ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26502024-11-28T01:43:46.562ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26512024-11-28T01:43:46.562ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26522024-11-28T01:43:46.563ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26532024-11-28T01:43:46.563ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26542024-11-28T01:43:46.564ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26552024-11-28T01:43:46.565ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26562024-11-28T01:43:46.565ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26572024-11-28T01:43:46.566ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26582024-11-28T01:43:46.566ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26592024-11-28T01:43:46.567ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26602024-11-28T01:43:46.568ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26612024-11-28T01:43:46.568ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26622024-11-28T01:43:46.569ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26632024-11-28T01:43:46.570ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26642024-11-28T01:43:46.570ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26652024-11-28T01:43:46.572ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26662024-11-28T01:43:46.573ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26672024-11-28T01:43:56.565ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26682024-11-28T01:43:56.566ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26692024-11-28T01:43:56.566ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26702024-11-28T01:43:56.567ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26712024-11-28T01:43:56.568ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26722024-11-28T01:43:56.568ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26732024-11-28T01:43:56.569ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26742024-11-28T01:43:56.569ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26752024-11-28T01:43:56.570ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26762024-11-28T01:43:56.571ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26772024-11-28T01:43:56.571ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26782024-11-28T01:43:56.572ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26792024-11-28T01:43:56.572ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26802024-11-28T01:43:56.574ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26812024-11-28T01:43:56.574ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26822024-11-28T01:43:56.575ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26832024-11-28T01:43:56.576ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26842024-11-28T01:43:56.577ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26852024-11-28T01:43:57.185ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:49453
26862024-11-28T01:43:57.185ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:49453 req_id = 421c1b6e-02e8-4e6a-9ccd-466959519463 response_code = 200 uri = /baseboard
26872024-11-28T01:44:06.569ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26882024-11-28T01:44:06.569ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26892024-11-28T01:44:06.570ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26902024-11-28T01:44:06.571ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26912024-11-28T01:44:06.571ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26922024-11-28T01:44:06.572ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26932024-11-28T01:44:06.572ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26942024-11-28T01:44:06.573ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26952024-11-28T01:44:06.573ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26962024-11-28T01:44:06.574ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26972024-11-28T01:44:06.575ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26982024-11-28T01:44:06.575ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
26992024-11-28T01:44:06.576ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27002024-11-28T01:44:06.577ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27012024-11-28T01:44:06.578ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27022024-11-28T01:44:06.578ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27032024-11-28T01:44:06.579ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27042024-11-28T01:44:06.580ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27052024-11-28T01:44:12.250ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48569
27062024-11-28T01:44:12.250ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48569 req_id = 56e3a1ae-8f54-4a67-a725-4fb55afabc4b response_code = 200 uri = /vpc-routes
27072024-11-28T01:44:12.314ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
27082024-11-28T01:44:12.314ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48569 req_id = ff18bad1-cbe8-49bb-b36b-c22febc0b68c response_code = 204 uri = /eip-gateways
27092024-11-28T01:44:12.328ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 278 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48569 req_id = 8b759af2-4596-47fe-bfcb-fb0b92c3db68 response_code = 204 uri = /vpc-routes
27102024-11-28T01:44:12.440ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36589
27112024-11-28T01:44:12.441ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36589 req_id = 7bcc53a3-cd5a-4a7d-8808-c9a4980bfad2 response_code = 200 uri = /vpc-routes
27122024-11-28T01:44:12.523ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
27132024-11-28T01:44:12.523ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:36589 req_id = f825483a-7922-4bf9-8753-550a6bfeb9de response_code = 204 uri = /eip-gateways
27142024-11-28T01:44:12.550ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33284
27152024-11-28T01:44:12.550ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 184 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33284 req_id = 6738b87d-8db1-46a2-821f-0207a94065eb response_code = 200 uri = /v2p
27162024-11-28T01:44:12.558ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:36589 req_id = 4af9b361-e6a3-435f-80e2-1dfe2c2d3b8a response_code = 204 uri = /vpc-routes
27172024-11-28T01:44:12.588ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45479
27182024-11-28T01:44:12.588ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45479 req_id = 1163584c-0721-42e0-aba9-27295b15d49a response_code = 200 uri = /v2p
27192024-11-28T01:44:12.612ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57761
27202024-11-28T01:44:12.612ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
27212024-11-28T01:44:12.661ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
27222024-11-28T01:44:12.736ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 124544 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:57761 req_id = a98f22fd-e630-40bf-b073-8cdfed9485cf response_code = 204 uri = /switch-ports
27232024-11-28T01:44:12.736ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49875
27242024-11-28T01:44:12.737ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36777
27252024-11-28T01:44:12.742ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4939 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57761 req_id = cabef41b-32a5-4af3-89b4-6ecf534ae78e response_code = 200 uri = /network-bootstore-config
27262024-11-28T01:44:12.746ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
27272024-11-28T01:44:12.747ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9961 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49875 req_id = d55b64fe-a230-437d-a885-05fffa134815 response_code = 200 uri = /vpc-routes
27282024-11-28T01:44:12.786ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
27292024-11-28T01:44:12.792ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
27302024-11-28T01:44:12.792ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 219 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49875 req_id = d5b6d367-cd72-477f-8021-3f8b9faa9d76 response_code = 204 uri = /eip-gateways
27312024-11-28T01:44:12.806ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 825 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49875 req_id = 1e463603-3c03-44c4-b237-11be5749198c response_code = 204 uri = /vpc-routes
27322024-11-28T01:44:12.852ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111214 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:36777 req_id = 29dba430-2706-4505-8af6-e8029c785f86 response_code = 204 uri = /switch-ports
27332024-11-28T01:44:12.853ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 156 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36777 req_id = 99984a3e-09dc-4599-9cd4-e98241dbec31 response_code = 200 uri = /network-bootstore-config
27342024-11-28T01:44:12.964ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36656
27352024-11-28T01:44:12.964ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36656 req_id = d76ebc65-69ec-4e0d-aa6a-810068175945 response_code = 200 uri = /v2p
27362024-11-28T01:44:13.005ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62827
27372024-11-28T01:44:13.005ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
27382024-11-28T01:44:13.052ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
27392024-11-28T01:44:13.119ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113962 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:62827 req_id = 63fe1aa1-6ff5-4604-9347-053f802d7df6 response_code = 204 uri = /switch-ports
27402024-11-28T01:44:13.119ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62827 req_id = 38a5f6e2-85f6-4941-8a33-e917bdd112dd response_code = 200 uri = /network-bootstore-config
27412024-11-28T01:44:16.572ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27422024-11-28T01:44:16.572ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27432024-11-28T01:44:16.573ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27442024-11-28T01:44:16.574ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27452024-11-28T01:44:16.574ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27462024-11-28T01:44:16.575ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27472024-11-28T01:44:16.575ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27482024-11-28T01:44:16.576ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27492024-11-28T01:44:16.576ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27502024-11-28T01:44:16.577ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27512024-11-28T01:44:16.578ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27522024-11-28T01:44:16.578ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27532024-11-28T01:44:16.579ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27542024-11-28T01:44:16.580ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27552024-11-28T01:44:16.581ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27562024-11-28T01:44:16.581ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27572024-11-28T01:44:16.583ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27582024-11-28T01:44:16.583ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27592024-11-28T01:44:26.576ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27602024-11-28T01:44:26.577ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27612024-11-28T01:44:26.578ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27622024-11-28T01:44:26.578ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27632024-11-28T01:44:26.579ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27642024-11-28T01:44:26.579ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27652024-11-28T01:44:26.580ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27662024-11-28T01:44:26.581ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27672024-11-28T01:44:26.581ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27682024-11-28T01:44:26.582ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27692024-11-28T01:44:26.582ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27702024-11-28T01:44:26.583ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27712024-11-28T01:44:26.584ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27722024-11-28T01:44:26.585ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27732024-11-28T01:44:26.586ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27742024-11-28T01:44:26.586ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27752024-11-28T01:44:26.587ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27762024-11-28T01:44:26.588ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27772024-11-28T01:44:27.218ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:64339
27782024-11-28T01:44:27.218ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 31 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:64339 req_id = 4f93707b-7f9d-408b-a21e-f6d36995737d response_code = 200 uri = /baseboard
27792024-11-28T01:44:36.581ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27802024-11-28T01:44:36.581ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27812024-11-28T01:44:36.582ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27822024-11-28T01:44:36.583ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27832024-11-28T01:44:36.583ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27842024-11-28T01:44:36.584ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27852024-11-28T01:44:36.584ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27862024-11-28T01:44:36.585ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27872024-11-28T01:44:36.586ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27882024-11-28T01:44:36.586ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27892024-11-28T01:44:36.587ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27902024-11-28T01:44:36.587ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27912024-11-28T01:44:36.588ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27922024-11-28T01:44:36.589ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27932024-11-28T01:44:36.590ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27942024-11-28T01:44:36.590ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27952024-11-28T01:44:36.592ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27962024-11-28T01:44:36.592ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
27972024-11-28T01:44:42.230ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35654
27982024-11-28T01:44:42.230ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 72 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35654 req_id = 87d37c4f-b6ea-4803-bfd5-541016c670f9 response_code = 200 uri = /vpc-routes
27992024-11-28T01:44:42.270ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64202
28002024-11-28T01:44:42.270ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64202 req_id = 1c84c049-2fea-4402-ae89-89c82088c492 response_code = 200 uri = /vpc-routes
28012024-11-28T01:44:42.286ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
28022024-11-28T01:44:42.286ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64202 req_id = dd0bb2e0-2567-4f81-8068-69eca8f5bde7 response_code = 204 uri = /eip-gateways
28032024-11-28T01:44:42.288ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
28042024-11-28T01:44:42.288ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 21 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35654 req_id = d3c24218-d297-401b-a989-15a511e22a6f response_code = 204 uri = /eip-gateways
28052024-11-28T01:44:42.298ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 164 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35654 req_id = 3116bd01-1f24-4244-af81-fd5327011116 response_code = 204 uri = /vpc-routes
28062024-11-28T01:44:42.336ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53627
28072024-11-28T01:44:42.336ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 159 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53627 req_id = e9108389-ef30-4bc6-bfbe-6d2dee02cd96 response_code = 200 uri = /v2p
28082024-11-28T01:44:42.341ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 168 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64202 req_id = 5d196eae-cf0f-4107-89ed-8132eea66775 response_code = 204 uri = /vpc-routes
28092024-11-28T01:44:42.412ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61048
28102024-11-28T01:44:42.412ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
28112024-11-28T01:44:42.453ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
28122024-11-28T01:44:42.520ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 107521 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:61048 req_id = 441cc7d4-f79b-433d-b71b-1699a9f926a1 response_code = 204 uri = /switch-ports
28132024-11-28T01:44:42.520ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52919
28142024-11-28T01:44:42.520ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52555
28152024-11-28T01:44:42.520ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 260 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61048 req_id = d4b5884f-7e50-4990-b1b4-41935124457f response_code = 200 uri = /network-bootstore-config
28162024-11-28T01:44:42.521ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
28172024-11-28T01:44:42.521ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52919 req_id = 0214ffc9-aaf5-4a85-8601-7a76cbc19535 response_code = 200 uri = /v2p
28182024-11-28T01:44:42.528ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49989
28192024-11-28T01:44:42.528ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49989 req_id = f3f64423-f963-4970-a6c2-a2eaa43dec64 response_code = 200 uri = /vpc-routes
28202024-11-28T01:44:42.565ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
28212024-11-28T01:44:42.572ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
28222024-11-28T01:44:42.572ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49989 req_id = abcc9bdb-e3e4-43d8-9b95-fa7efc4b4ca2 response_code = 204 uri = /eip-gateways
28232024-11-28T01:44:42.583ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1008 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49989 req_id = 6f20ca27-cd3b-4c83-a594-b4bf184b52b4 response_code = 204 uri = /vpc-routes
28242024-11-28T01:44:42.620ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43776
28252024-11-28T01:44:42.621ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 241 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43776 req_id = 29e079fe-5c4b-4b77-b75d-a91050354215 response_code = 200 uri = /v2p
28262024-11-28T01:44:42.629ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108583 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:52555 req_id = f1c50268-08f3-4e39-a77e-087651becb84 response_code = 204 uri = /switch-ports
28272024-11-28T01:44:42.629ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52555 req_id = 7930123c-5dd5-49cc-96cb-cb1435a72816 response_code = 200 uri = /network-bootstore-config
28282024-11-28T01:44:42.633ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57554
28292024-11-28T01:44:42.633ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
28302024-11-28T01:44:42.671ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
28312024-11-28T01:44:42.737ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103929 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:57554 req_id = a427eabf-d7a8-470e-8eea-73f6b3c3c8a4 response_code = 204 uri = /switch-ports
28322024-11-28T01:44:42.737ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57554 req_id = ff327a24-5e03-4612-89a6-e27713e2b2d6 response_code = 200 uri = /network-bootstore-config
28332024-11-28T01:44:46.585ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28342024-11-28T01:44:46.586ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28352024-11-28T01:44:46.586ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28362024-11-28T01:44:46.587ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28372024-11-28T01:44:46.587ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28382024-11-28T01:44:46.588ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28392024-11-28T01:44:46.589ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28402024-11-28T01:44:46.589ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28412024-11-28T01:44:46.590ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28422024-11-28T01:44:46.590ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28432024-11-28T01:44:46.591ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28442024-11-28T01:44:46.592ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28452024-11-28T01:44:46.592ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28462024-11-28T01:44:46.594ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28472024-11-28T01:44:46.594ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28482024-11-28T01:44:46.595ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28492024-11-28T01:44:46.596ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28502024-11-28T01:44:46.597ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28512024-11-28T01:44:56.588ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28522024-11-28T01:44:56.589ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28532024-11-28T01:44:56.590ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28542024-11-28T01:44:56.590ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28552024-11-28T01:44:56.591ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28562024-11-28T01:44:56.591ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28572024-11-28T01:44:56.592ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28582024-11-28T01:44:56.593ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28592024-11-28T01:44:56.593ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28602024-11-28T01:44:56.594ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28612024-11-28T01:44:56.594ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28622024-11-28T01:44:56.595ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28632024-11-28T01:44:56.596ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28642024-11-28T01:44:56.597ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28652024-11-28T01:44:56.597ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28662024-11-28T01:44:56.598ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28672024-11-28T01:44:56.599ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28682024-11-28T01:44:56.600ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28692024-11-28T01:44:57.252ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:34443
28702024-11-28T01:44:57.252ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 32 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:34443 req_id = 864074de-9de5-4eaf-acf6-9679d8640f36 response_code = 200 uri = /baseboard
28712024-11-28T01:45:06.592ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28722024-11-28T01:45:06.593ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28732024-11-28T01:45:06.594ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28742024-11-28T01:45:06.594ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28752024-11-28T01:45:06.595ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28762024-11-28T01:45:06.595ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28772024-11-28T01:45:06.596ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28782024-11-28T01:45:06.597ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28792024-11-28T01:45:06.597ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28802024-11-28T01:45:06.598ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28812024-11-28T01:45:06.598ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28822024-11-28T01:45:06.599ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28832024-11-28T01:45:06.600ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28842024-11-28T01:45:06.601ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28852024-11-28T01:45:06.602ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28862024-11-28T01:45:06.603ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28872024-11-28T01:45:06.604ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28882024-11-28T01:45:06.605ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
28892024-11-28T01:45:12.407ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51251
28902024-11-28T01:45:12.407ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 75 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51251 req_id = 13e87b13-5017-4dbd-9462-6542f7a9df45 response_code = 200 uri = /vpc-routes
28912024-11-28T01:45:12.464ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37493
28922024-11-28T01:45:12.464ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37493 req_id = ac4f9165-245e-487f-bebb-c86266157adf response_code = 200 uri = /vpc-routes
28932024-11-28T01:45:12.468ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
28942024-11-28T01:45:12.468ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51251 req_id = fad1c3e1-d895-4ffe-a158-32e07181d7de response_code = 204 uri = /eip-gateways
28952024-11-28T01:45:12.484ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 228 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51251 req_id = 82a37382-8c05-4522-8f9e-ca05b8089978 response_code = 204 uri = /vpc-routes
28962024-11-28T01:45:12.515ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
28972024-11-28T01:45:12.515ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37493 req_id = 09ba4588-faa7-4f34-8eff-a4c6886866e2 response_code = 204 uri = /eip-gateways
28982024-11-28T01:45:12.525ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 203 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37493 req_id = 703ab09a-1b03-4a37-8209-39b54938776f response_code = 204 uri = /vpc-routes
28992024-11-28T01:45:12.607ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48470
29002024-11-28T01:45:12.607ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
29012024-11-28T01:45:12.607ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
29022024-11-28T01:45:12.607ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/omicron-zones.json file = common/src/ledger.rs:177
29032024-11-28T01:45:12.608ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/omicron-zones.json file = common/src/ledger.rs:177
29042024-11-28T01:45:12.615ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36645
29052024-11-28T01:45:12.615ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 267 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36645 req_id = 0c979c3b-c595-48b1-8cb3-90925048e8a1 response_code = 200 uri = /v2p
29062024-11-28T01:45:12.615ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46643
29072024-11-28T01:45:12.615ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46643 req_id = 36fca0dc-7a5d-4938-9d91-a5d5a9130d7e response_code = 200 uri = /vpc-routes
29082024-11-28T01:45:12.627ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29092024-11-28T01:45:12.627ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee
29102024-11-28T01:45:12.671ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29112024-11-28T01:45:12.672ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8164ba35-0eff-436a-9b6b-80912419e510
29122024-11-28T01:45:12.689ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37826
29132024-11-28T01:45:12.691ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
29142024-11-28T01:45:12.691ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
29152024-11-28T01:45:12.691ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/omicron-zones.json file = common/src/ledger.rs:177
29162024-11-28T01:45:12.699ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
29172024-11-28T01:45:12.699ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 851 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46643 req_id = 442c5e33-30fa-4e66-8624-1a6b8469d3b3 response_code = 204 uri = /eip-gateways
29182024-11-28T01:45:12.699ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/omicron-zones.json file = common/src/ledger.rs:177
29192024-11-28T01:45:12.707ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54250
29202024-11-28T01:45:12.707ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
29212024-11-28T01:45:12.714ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29222024-11-28T01:45:12.715ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_0176efd8-6a11-46a9-8bcd-a97e447fcb50
29232024-11-28T01:45:12.725ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 898 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46643 req_id = 41235ad1-8912-42eb-8b15-7dfd112d28e0 response_code = 204 uri = /vpc-routes
29242024-11-28T01:45:12.737ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29252024-11-28T01:45:12.737ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee
29262024-11-28T01:45:12.746ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37987
29272024-11-28T01:45:12.746ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 286 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37987 req_id = a940a12b-3d2e-463f-936b-ab2d40248c8a response_code = 200 uri = /v2p
29282024-11-28T01:45:12.749ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60002
29292024-11-28T01:45:12.750ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 7517eed1-3882-4d66-a665-f02c99641d63, kind: Service { id: b555dc2a-491b-4b5c-b5d7-24801f82c048 }, name: Name("external-dns-b555dc2a-491b-4b5c-b5d7-24801f82c048"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 105])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bdbab4fe-05a5-497c-92d5-51ab355ee8a2, kind: Service { id: 7b4111aa-1a34-450f-8a24-b1db940782bf }, name: Name("external-dns-7b4111aa-1a34-450f-8a24-b1db940782bf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 193, 141])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 57c41de5-57cc-4ec0-b2a4-ec18275db374, kind: Service { id: 75369688-7093-4009-81fe-7452642ec8ea }, name: Name("nexus-75369688-7093-4009-81fe-7452642ec8ea"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5d808073-b6dd-4352-8286-483c26ccb36e, kind: Service { id: 80d50149-adaa-4067-b457-5cd986e26bb3 }, name: Name("nexus-80d50149-adaa-4067-b457-5cd986e26bb3"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c047ac03-3e6b-4894-a737-123382b38a57, kind: Service { id: 0cdd90ac-34d6-41d9-bd04-18f3635adaf7 }, name: Name("nexus-0cdd90ac-34d6-41d9-bd04-18f3635adaf7"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
29302024-11-28T01:45:12.755ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
29312024-11-28T01:45:12.756ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
29322024-11-28T01:45:12.756ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
29332024-11-28T01:45:12.756ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
29342024-11-28T01:45:12.756ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
29352024-11-28T01:45:12.756ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
29362024-11-28T01:45:12.756ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6824 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:60002 req_id = c7483795-7c1d-4cba-a03c-5cf5bf5bd100 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
29372024-11-28T01:45:12.768ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
29382024-11-28T01:45:12.783ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29392024-11-28T01:45:12.783ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8164ba35-0eff-436a-9b6b-80912419e510
29402024-11-28T01:45:12.808ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29412024-11-28T01:45:12.808ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_beaaa347-4478-48b4-b888-a5873241ded0
29422024-11-28T01:45:12.830ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33828
29432024-11-28T01:45:12.842ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29442024-11-28T01:45:12.842ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_0176efd8-6a11-46a9-8bcd-a97e447fcb50
29452024-11-28T01:45:12.859ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29462024-11-28T01:45:12.859ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_216077bb-f8f0-4fbf-8aa0-1db2249e65da
29472024-11-28T01:45:12.866ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
29482024-11-28T01:45:12.866ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 159276 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:54250 req_id = f571555d-78fd-4063-81ba-45a49937af6e response_code = 204 uri = /switch-ports
29492024-11-28T01:45:12.871ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 657 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54250 req_id = 06393a54-6408-4d5a-b4aa-03b302cc14d5 response_code = 200 uri = /network-bootstore-config
29502024-11-28T01:45:12.890ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29512024-11-28T01:45:12.891ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_977ab5dc-c743-4525-b172-ecf8a9034abe
29522024-11-28T01:45:12.897ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56640
29532024-11-28T01:45:12.906ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64231
29542024-11-28T01:45:12.906ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 221 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64231 req_id = a563ef84-81be-42dc-92ac-ada82e0187dc response_code = 200 uri = /v2p
29552024-11-28T01:45:12.908ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29562024-11-28T01:45:12.908ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_beaaa347-4478-48b4-b888-a5873241ded0
29572024-11-28T01:45:12.917ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
29582024-11-28T01:45:12.927ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37397
29592024-11-28T01:45:12.931ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
29602024-11-28T01:45:12.931ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
29612024-11-28T01:45:12.931ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29622024-11-28T01:45:12.931ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_2fcc59ee-335a-4c80-bbcf-e66263237ec3
29632024-11-28T01:45:12.931ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/config/omicron-zones.json file = common/src/ledger.rs:177
29642024-11-28T01:45:12.951ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/config/omicron-zones.json file = common/src/ledger.rs:177
29652024-11-28T01:45:12.962ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29662024-11-28T01:45:12.962ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_216077bb-f8f0-4fbf-8aa0-1db2249e65da
29672024-11-28T01:45:12.979ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: a6e234b1-bf4c-4b91-92ee-604781b835ee (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29682024-11-28T01:45:12.979ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_a6e234b1-bf4c-4b91-92ee-604781b835ee
29692024-11-28T01:45:13.001ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29702024-11-28T01:45:13.006ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c7ca3a79-57f5-4e47-8807-03bd67966814
29712024-11-28T01:45:13.006ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
29722024-11-28T01:45:13.007ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 171137 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:33828 req_id = a72b3345-f471-4d15-9f24-fc96e3e53563 response_code = 204 uri = /switch-ports
29732024-11-28T01:45:13.021ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6827 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33828 req_id = 38dae8e5-bfd1-4114-9077-718b3d760e53 response_code = 200 uri = /network-bootstore-config
29742024-11-28T01:45:13.031ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35274
29752024-11-28T01:45:13.031ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29762024-11-28T01:45:13.031ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_977ab5dc-c743-4525-b172-ecf8a9034abe
29772024-11-28T01:45:13.044ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 7517eed1-3882-4d66-a665-f02c99641d63, kind: Service { id: b555dc2a-491b-4b5c-b5d7-24801f82c048 }, name: Name("external-dns-b555dc2a-491b-4b5c-b5d7-24801f82c048"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 105])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bdbab4fe-05a5-497c-92d5-51ab355ee8a2, kind: Service { id: 7b4111aa-1a34-450f-8a24-b1db940782bf }, name: Name("external-dns-7b4111aa-1a34-450f-8a24-b1db940782bf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 193, 141])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 57c41de5-57cc-4ec0-b2a4-ec18275db374, kind: Service { id: 75369688-7093-4009-81fe-7452642ec8ea }, name: Name("nexus-75369688-7093-4009-81fe-7452642ec8ea"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5d808073-b6dd-4352-8286-483c26ccb36e, kind: Service { id: 80d50149-adaa-4067-b457-5cd986e26bb3 }, name: Name("nexus-80d50149-adaa-4067-b457-5cd986e26bb3"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c047ac03-3e6b-4894-a737-123382b38a57, kind: Service { id: 0cdd90ac-34d6-41d9-bd04-18f3635adaf7 }, name: Name("nexus-0cdd90ac-34d6-41d9-bd04-18f3635adaf7"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
29782024-11-28T01:45:13.044ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
29792024-11-28T01:45:13.044ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
29802024-11-28T01:45:13.044ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
29812024-11-28T01:45:13.044ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
29822024-11-28T01:45:13.044ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
29832024-11-28T01:45:13.044ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
29842024-11-28T01:45:13.051ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8164ba35-0eff-436a-9b6b-80912419e510 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29852024-11-28T01:45:13.051ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8164ba35-0eff-436a-9b6b-80912419e510
29862024-11-28T01:45:13.051ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 19979 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35274 req_id = b43dea56-7cfe-4f98-b341-59e46914cd82 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
29872024-11-28T01:45:13.062ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
29882024-11-28T01:45:13.064ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57794
29892024-11-28T01:45:13.069ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29902024-11-28T01:45:13.070ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_75598bf6-d28b-4334-94db-398fc2a8b539
29912024-11-28T01:45:13.085ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 7517eed1-3882-4d66-a665-f02c99641d63, kind: Service { id: b555dc2a-491b-4b5c-b5d7-24801f82c048 }, name: Name("external-dns-b555dc2a-491b-4b5c-b5d7-24801f82c048"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 105])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bdbab4fe-05a5-497c-92d5-51ab355ee8a2, kind: Service { id: 7b4111aa-1a34-450f-8a24-b1db940782bf }, name: Name("external-dns-7b4111aa-1a34-450f-8a24-b1db940782bf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 193, 141])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 57c41de5-57cc-4ec0-b2a4-ec18275db374, kind: Service { id: 75369688-7093-4009-81fe-7452642ec8ea }, name: Name("nexus-75369688-7093-4009-81fe-7452642ec8ea"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 229, 184])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5d808073-b6dd-4352-8286-483c26ccb36e, kind: Service { id: 80d50149-adaa-4067-b457-5cd986e26bb3 }, name: Name("nexus-80d50149-adaa-4067-b457-5cd986e26bb3"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 213, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c047ac03-3e6b-4894-a737-123382b38a57, kind: Service { id: 0cdd90ac-34d6-41d9-bd04-18f3635adaf7 }, name: Name("nexus-0cdd90ac-34d6-41d9-bd04-18f3635adaf7"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 201, 190])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
29922024-11-28T01:45:13.085ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
29932024-11-28T01:45:13.085ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
29942024-11-28T01:45:13.085ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
29952024-11-28T01:45:13.085ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
29962024-11-28T01:45:13.085ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
29972024-11-28T01:45:13.085ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
29982024-11-28T01:45:13.085ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6534 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57794 req_id = 1e91c42c-66f8-4909-a9f5-84db7224dd35 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
29992024-11-28T01:45:13.089ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
30002024-11-28T01:45:13.089ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_2fcc59ee-335a-4c80-bbcf-e66263237ec3
30012024-11-28T01:45:13.099ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 491151 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48470 req_id = 5c2f924b-94a3-4c6a-842e-54434dcec384 response_code = 200 uri = /inventory
30022024-11-28T01:45:13.104ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0176efd8-6a11-46a9-8bcd-a97e447fcb50 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
30032024-11-28T01:45:13.104ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_0176efd8-6a11-46a9-8bcd-a97e447fcb50
30042024-11-28T01:45:13.123ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
30052024-11-28T01:45:13.123ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c7ca3a79-57f5-4e47-8807-03bd67966814
30062024-11-28T01:45:13.140ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: beaaa347-4478-48b4-b888-a5873241ded0 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
30072024-11-28T01:45:13.140ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_beaaa347-4478-48b4-b888-a5873241ded0
30082024-11-28T01:45:13.145ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 217574 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:37397 req_id = 9299ac58-c875-4349-b2e1-98fb597eefa1 response_code = 204 uri = /switch-ports
30092024-11-28T01:45:13.146ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 314 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37397 req_id = b6b89495-9d46-4d2e-a76b-fc6346dd612a response_code = 200 uri = /network-bootstore-config
30102024-11-28T01:45:13.176ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 216077bb-f8f0-4fbf-8aa0-1db2249e65da (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
30112024-11-28T01:45:13.177ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_216077bb-f8f0-4fbf-8aa0-1db2249e65da
30122024-11-28T01:45:13.193ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
30132024-11-28T01:45:13.193ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_75598bf6-d28b-4334-94db-398fc2a8b539
30142024-11-28T01:45:13.213ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 523533 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37826 req_id = 1e329bb4-e5d7-4747-a572-553199a88a91 response_code = 200 uri = /inventory
30152024-11-28T01:45:13.218ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 977ab5dc-c743-4525-b172-ecf8a9034abe (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
30162024-11-28T01:45:13.218ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_977ab5dc-c743-4525-b172-ecf8a9034abe
30172024-11-28T01:45:13.254ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 2fcc59ee-335a-4c80-bbcf-e66263237ec3 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
30182024-11-28T01:45:13.255ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_2fcc59ee-335a-4c80-bbcf-e66263237ec3
30192024-11-28T01:45:13.286ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c7ca3a79-57f5-4e47-8807-03bd67966814 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
30202024-11-28T01:45:13.286ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c7ca3a79-57f5-4e47-8807-03bd67966814
30212024-11-28T01:45:13.317ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 75598bf6-d28b-4334-94db-398fc2a8b539 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
30222024-11-28T01:45:13.317ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_75598bf6-d28b-4334-94db-398fc2a8b539
30232024-11-28T01:45:13.333ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 436381 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56640 req_id = 747498b5-f187-4adc-b21e-4105115d6191 response_code = 200 uri = /inventory
30242024-11-28T01:45:16.596ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30252024-11-28T01:45:16.597ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30262024-11-28T01:45:16.598ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30272024-11-28T01:45:16.599ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30282024-11-28T01:45:16.599ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30292024-11-28T01:45:16.600ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30302024-11-28T01:45:16.600ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30312024-11-28T01:45:16.601ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30322024-11-28T01:45:16.602ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30332024-11-28T01:45:16.602ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30342024-11-28T01:45:16.603ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30352024-11-28T01:45:16.604ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30362024-11-28T01:45:16.604ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30372024-11-28T01:45:16.606ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30382024-11-28T01:45:16.606ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30392024-11-28T01:45:16.607ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30402024-11-28T01:45:16.608ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30412024-11-28T01:45:16.609ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30422024-11-28T01:45:26.600ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30432024-11-28T01:45:26.601ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30442024-11-28T01:45:26.601ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30452024-11-28T01:45:26.602ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30462024-11-28T01:45:26.603ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30472024-11-28T01:45:26.603ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30482024-11-28T01:45:26.604ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30492024-11-28T01:45:26.604ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30502024-11-28T01:45:26.605ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30512024-11-28T01:45:26.606ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30522024-11-28T01:45:26.606ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30532024-11-28T01:45:26.607ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30542024-11-28T01:45:26.607ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30552024-11-28T01:45:26.609ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30562024-11-28T01:45:26.609ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30572024-11-28T01:45:26.610ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30582024-11-28T01:45:26.611ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30592024-11-28T01:45:26.612ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30602024-11-28T01:45:27.284ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:36631
30612024-11-28T01:45:27.284ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 31 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:36631 req_id = ba3d6da3-4b8f-4209-a63a-fb9d103e1f4c response_code = 200 uri = /baseboard
30622024-11-28T01:45:36.604ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30632024-11-28T01:45:36.605ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30642024-11-28T01:45:36.606ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30652024-11-28T01:45:36.606ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30662024-11-28T01:45:36.607ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30672024-11-28T01:45:36.607ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30682024-11-28T01:45:36.608ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30692024-11-28T01:45:36.609ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30702024-11-28T01:45:36.609ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30712024-11-28T01:45:36.610ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30722024-11-28T01:45:36.611ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30732024-11-28T01:45:36.611ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30742024-11-28T01:45:36.612ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30752024-11-28T01:45:36.613ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30762024-11-28T01:45:36.614ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30772024-11-28T01:45:36.614ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30782024-11-28T01:45:36.616ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30792024-11-28T01:45:36.617ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
30802024-11-28T01:45:42.108ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41837
30812024-11-28T01:45:42.108ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41837 req_id = 69433172-0366-462c-b196-3277be25407c response_code = 200 uri = /vpc-routes
30822024-11-28T01:45:42.142ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
30832024-11-28T01:45:42.142ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:41837 req_id = 1d98df48-2193-4088-b8d8-3d67ae9601a3 response_code = 204 uri = /eip-gateways
30842024-11-28T01:45:42.153ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 254 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:41837 req_id = 75ecf14a-d799-42df-95d0-ec0c4815fb1e response_code = 204 uri = /vpc-routes
30852024-11-28T01:45:42.262ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64864
30862024-11-28T01:45:42.262ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64864 req_id = 1d72a10f-1a1f-4dfb-8d37-e81249dd3713 response_code = 200 uri = /vpc-routes
30872024-11-28T01:45:42.308ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51479
30882024-11-28T01:45:42.309ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 170 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51479 req_id = 6aa1768c-f606-4fac-923b-47a9746133e4 response_code = 200 uri = /v2p
30892024-11-28T01:45:42.319ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
30902024-11-28T01:45:42.319ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64864 req_id = 966d926b-5b6e-4540-914b-73889da4582b response_code = 204 uri = /eip-gateways
30912024-11-28T01:45:42.329ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 219 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64864 req_id = cbc2d5f3-9a07-47e6-88cd-61ede1af6348 response_code = 204 uri = /vpc-routes
30922024-11-28T01:45:42.332ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40249
30932024-11-28T01:45:42.332ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
30942024-11-28T01:45:42.390ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
30952024-11-28T01:45:42.481ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 149261 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:40249 req_id = 4b5fbf26-ff3b-4203-9e12-fb19fe14aeec response_code = 204 uri = /switch-ports
30962024-11-28T01:45:42.482ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51114
30972024-11-28T01:45:42.482ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57804
30982024-11-28T01:45:42.482ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 207 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57804 req_id = 3be9bef3-827f-4192-a36e-b5b727756874 response_code = 200 uri = /v2p
30992024-11-28T01:45:42.482ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
31002024-11-28T01:45:42.488ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5790 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40249 req_id = b4dea73b-3a32-48af-9abc-f7d430ac6b09 response_code = 200 uri = /network-bootstore-config
31012024-11-28T01:45:42.524ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
31022024-11-28T01:45:42.529ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63257
31032024-11-28T01:45:42.529ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63257 req_id = 7a3abdc1-95c6-495c-b981-27708abf0260 response_code = 200 uri = /vpc-routes
31042024-11-28T01:45:42.576ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
31052024-11-28T01:45:42.576ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 583 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63257 req_id = e3f78965-4be6-488f-86f0-068301c2deac response_code = 204 uri = /eip-gateways
31062024-11-28T01:45:42.582ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42453
31072024-11-28T01:45:42.582ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42453 req_id = bfd0782a-5207-4a50-a03d-0a9bd8d347b8 response_code = 200 uri = /v2p
31082024-11-28T01:45:42.585ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103329 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:51114 req_id = 37844338-444f-4567-81e9-0861855a4416 response_code = 204 uri = /switch-ports
31092024-11-28T01:45:42.585ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51114 req_id = 8e023338-29e5-499d-992b-9921493985b2 response_code = 200 uri = /network-bootstore-config
31102024-11-28T01:45:42.586ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63257 req_id = 12304598-348a-4256-ac10-48c0a8608ef5 response_code = 204 uri = /vpc-routes
31112024-11-28T01:45:42.635ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64019
31122024-11-28T01:45:42.635ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
31132024-11-28T01:45:42.673ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
31142024-11-28T01:45:42.732ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97206 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:64019 req_id = 58c23d7f-d3f3-492f-8a21-f6989a319d51 response_code = 204 uri = /switch-ports
31152024-11-28T01:45:42.733ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64019 req_id = f4cec5d9-8a95-4f02-93eb-7d8887fd5750 response_code = 200 uri = /network-bootstore-config
31162024-11-28T01:45:46.609ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31172024-11-28T01:45:46.609ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31182024-11-28T01:45:46.610ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31192024-11-28T01:45:46.611ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31202024-11-28T01:45:46.611ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31212024-11-28T01:45:46.612ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31222024-11-28T01:45:46.612ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31232024-11-28T01:45:46.613ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31242024-11-28T01:45:46.614ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31252024-11-28T01:45:46.614ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31262024-11-28T01:45:46.615ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31272024-11-28T01:45:46.616ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31282024-11-28T01:45:46.616ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31292024-11-28T01:45:46.618ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31302024-11-28T01:45:46.618ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31312024-11-28T01:45:46.619ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31322024-11-28T01:45:46.620ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31332024-11-28T01:45:46.621ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31342024-11-28T01:45:56.613ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31352024-11-28T01:45:56.614ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31362024-11-28T01:45:56.614ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31372024-11-28T01:45:56.615ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31382024-11-28T01:45:56.615ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31392024-11-28T01:45:56.616ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31402024-11-28T01:45:56.617ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31412024-11-28T01:45:56.617ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31422024-11-28T01:45:56.618ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31432024-11-28T01:45:56.618ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31442024-11-28T01:45:56.619ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31452024-11-28T01:45:56.620ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31462024-11-28T01:45:56.620ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31472024-11-28T01:45:56.622ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31482024-11-28T01:45:56.622ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31492024-11-28T01:45:56.623ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31502024-11-28T01:45:56.624ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31512024-11-28T01:45:56.625ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31522024-11-28T01:45:57.319ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:42838
31532024-11-28T01:45:57.319ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:42838 req_id = 706dd7d6-3d91-444e-9fcc-962b46ec68de response_code = 200 uri = /baseboard
31542024-11-28T01:46:06.617ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31552024-11-28T01:46:06.618ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31562024-11-28T01:46:06.619ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31572024-11-28T01:46:06.619ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31582024-11-28T01:46:06.620ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31592024-11-28T01:46:06.620ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31602024-11-28T01:46:06.621ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31612024-11-28T01:46:06.622ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31622024-11-28T01:46:06.622ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31632024-11-28T01:46:06.623ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31642024-11-28T01:46:06.623ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31652024-11-28T01:46:06.624ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31662024-11-28T01:46:06.625ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31672024-11-28T01:46:06.626ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31682024-11-28T01:46:06.626ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31692024-11-28T01:46:06.627ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31702024-11-28T01:46:06.628ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31712024-11-28T01:46:06.629ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
31722024-11-28T01:46:12.349ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62780
31732024-11-28T01:46:12.349ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62780 req_id = 79ebba76-efed-44bf-afde-0f884e4c6ea9 response_code = 200 uri = /vpc-routes
31742024-11-28T01:46:12.430ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
31752024-11-28T01:46:12.430ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:62780 req_id = a726b755-8abb-4573-8c95-2fc5431899b0 response_code = 204 uri = /eip-gateways
31762024-11-28T01:46:12.440ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41624
31772024-11-28T01:46:12.440ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 39 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41624 req_id = 831c31b4-679d-49d9-9dde-8901edd2b4b9 response_code = 200 uri = /vpc-routes
31782024-11-28T01:46:12.443ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 201 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:62780 req_id = 02a69d51-17d0-48e6-bcc3-0e0994399536 response_code = 204 uri = /vpc-routes
31792024-11-28T01:46:12.499ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
31802024-11-28T01:46:12.499ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41624 req_id = 7c56f09e-1444-47b5-96e0-4dafbffba71f response_code = 204 uri = /eip-gateways
31812024-11-28T01:46:12.514ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 183 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41624 req_id = 83076407-d742-4e82-bc87-eb0829ee1128 response_code = 204 uri = /vpc-routes
31822024-11-28T01:46:12.546ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38834
31832024-11-28T01:46:12.547ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
31842024-11-28T01:46:12.597ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
31852024-11-28T01:46:12.668ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 121520 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:38834 req_id = ce2bebf7-f527-4323-a1f2-9a9f5222d399 response_code = 204 uri = /switch-ports
31862024-11-28T01:46:12.668ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37618
31872024-11-28T01:46:12.668ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35803
31882024-11-28T01:46:12.668ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:65451
31892024-11-28T01:46:12.669ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 331 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38834 req_id = 669e8f14-1518-44af-bcc4-c013dc0c9511 response_code = 200 uri = /network-bootstore-config
31902024-11-28T01:46:12.669ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 302 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37618 req_id = 87ad488e-923d-4a78-880f-c746be92909c response_code = 200 uri = /v2p
31912024-11-28T01:46:12.669ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 248 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:65451 req_id = bd20d2bf-51ae-4634-9fad-f918da38118c response_code = 200 uri = /v2p
31922024-11-28T01:46:12.669ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
31932024-11-28T01:46:12.717ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
31942024-11-28T01:46:12.735ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61275
31952024-11-28T01:46:12.736ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 158 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61275 req_id = cebd6195-d570-4bb5-923e-279aeaaaf1a5 response_code = 200 uri = /vpc-routes
31962024-11-28T01:46:12.793ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
31972024-11-28T01:46:12.794ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 170 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:61275 req_id = e1565ac7-3b2c-4431-939d-377d1dba67fa response_code = 204 uri = /eip-gateways
31982024-11-28T01:46:12.804ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 134897 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:35803 req_id = 1d984af7-c323-41ce-bfbd-a844ef66f0c8 response_code = 204 uri = /switch-ports
31992024-11-28T01:46:12.804ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35803 req_id = cb984141-9947-4887-96fd-ebad50863dd1 response_code = 200 uri = /network-bootstore-config
32002024-11-28T01:46:12.808ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 291 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:61275 req_id = ca63b827-7621-49a9-b902-22f3ffd8671b response_code = 204 uri = /vpc-routes
32012024-11-28T01:46:12.857ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:32984
32022024-11-28T01:46:12.858ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
32032024-11-28T01:46:12.902ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
32042024-11-28T01:46:12.963ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105174 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:32984 req_id = 59c012b1-7851-4c02-922a-d21965b0dfa9 response_code = 204 uri = /switch-ports
32052024-11-28T01:46:12.963ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64951
32062024-11-28T01:46:12.963ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 182 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64951 req_id = acbbb121-5194-4860-9831-0b60891f69d5 response_code = 200 uri = /v2p
32072024-11-28T01:46:12.963ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 203 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:32984 req_id = a906fd85-c7f0-4701-9ddb-d689f4da397e response_code = 200 uri = /network-bootstore-config
32082024-11-28T01:46:16.621ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32092024-11-28T01:46:16.622ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32102024-11-28T01:46:16.622ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32112024-11-28T01:46:16.623ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32122024-11-28T01:46:16.623ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32132024-11-28T01:46:16.624ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32142024-11-28T01:46:16.625ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32152024-11-28T01:46:16.625ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32162024-11-28T01:46:16.626ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32172024-11-28T01:46:16.626ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32182024-11-28T01:46:16.627ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32192024-11-28T01:46:16.627ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32202024-11-28T01:46:16.628ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32212024-11-28T01:46:16.629ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32222024-11-28T01:46:16.630ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32232024-11-28T01:46:16.630ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32242024-11-28T01:46:16.632ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32252024-11-28T01:46:16.633ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32262024-11-28T01:46:26.625ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32272024-11-28T01:46:26.626ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32282024-11-28T01:46:26.626ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32292024-11-28T01:46:26.627ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32302024-11-28T01:46:26.627ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32312024-11-28T01:46:26.628ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32322024-11-28T01:46:26.629ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32332024-11-28T01:46:26.629ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32342024-11-28T01:46:26.630ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32352024-11-28T01:46:26.630ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32362024-11-28T01:46:26.632ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32372024-11-28T01:46:26.632ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32382024-11-28T01:46:26.633ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32392024-11-28T01:46:26.634ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32402024-11-28T01:46:26.635ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32412024-11-28T01:46:26.635ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32422024-11-28T01:46:26.637ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32432024-11-28T01:46:26.638ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32442024-11-28T01:46:27.354ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:63562
32452024-11-28T01:46:27.354ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 32 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:63562 req_id = 4987c9d6-d27b-4427-aae3-237b444c0cbe response_code = 200 uri = /baseboard
32462024-11-28T01:46:29.278ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55420
32472024-11-28T01:46:29.278ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 202 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55420 req_id = ced53701-7ce5-4c1a-bdfa-d98c3eef8391 response_code = 200 uri = /v2p
32482024-11-28T01:46:29.278ZINFOSledAgent (PortManager): Mapping virtual NIC to physical host file = illumos-utils/src/opte/port_manager.rs:874 mapping = VirtualNetworkInterfaceHost { virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), physical_host_ip: fd00:1122:3344:101::1, vni: Vni(10251726) }
32492024-11-28T01:46:29.278ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55420 req_id = 95100173-1c75-4777-a636-01126e332530 response_code = 204 uri = /v2p
32502024-11-28T01:46:29.367ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53770
32512024-11-28T01:46:29.372ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:555 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 9eb506bc-aa76-4401-bb2f-7c5bef9772ab, kind: Instance { id: eab2ccae-1246-4353-8b8d-c87c0502905b }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10251726), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.1.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.1.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 9eb506bc-aa76-4401-bb2f-7c5bef9772ab, kind: Instance { id: eab2ccae-1246-4353-8b8d-c87c0502905b }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10251726), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 9eb506bc-aa76-4401-bb2f-7c5bef9772ab, kind: Instance { id: eab2ccae-1246-4353-8b8d-c87c0502905b }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10251726), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(10251726))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 9eb506bc-aa76-4401-bb2f-7c5bef9772ab, kind: Instance { id: eab2ccae-1246-4353-8b8d-c87c0502905b }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10251726), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-3074993bd6e", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 834f6ede-be0e-4346-bf10-23744ed73332, read_only_parent: Some(Volume { block_size: 512, id: 1c2a4007-d32b-4141-8b9f-869b3243cd4f, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: f27e70fb-6921-4977-ac22-b2cf4c62b505, key: Some("kirjE5IKC4cmQ8QhZfU8V+GQKkmMB7g0qZU1CAVCOjo="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19001", "[fd00:1122:3344:101::19]:19001", "[fd00:1122:3344:101::18]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 834f6ede-be0e-4346-bf10-23744ed73332, key: Some("FJ9v4dVM4tYTRICfbqssrg12GPO7vKN1APmfBmuK2CE="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::16]:19001", "[fd00:1122:3344:101::13]:19001", "[fd00:1122:3344:101::1a]:19000"] } }] } }], boot_settings: Some(BootSettings { order: [BootOrderEntry { name: "disk-3074993bd6e" }] }), cloud_init_bytes: Some(..) } instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b metadata = InstanceMetadata { silo_id: bf218105-d1ce-4710-a3e5-44b53cababe2, project_id: f0e43e9c-63c9-45af-914c-90cb29f357e7 } migration_id = None propolis_addr = [fd00:1122:3344:101::1:0]:12400 propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-11-28T01:46:29.367285621Z }
32522024-11-28T01:46:29.372ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:588 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b migration_id = None propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32532024-11-28T01:46:29.373ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:1281 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b migration_id = None propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 9eb506bc-aa76-4401-bb2f-7c5bef9772ab, kind: Instance { id: eab2ccae-1246-4353-8b8d-c87c0502905b }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10251726), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.1.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.1.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 9eb506bc-aa76-4401-bb2f-7c5bef9772ab, kind: Instance { id: eab2ccae-1246-4353-8b8d-c87c0502905b }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10251726), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 9eb506bc-aa76-4401-bb2f-7c5bef9772ab, kind: Instance { id: eab2ccae-1246-4353-8b8d-c87c0502905b }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10251726), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(10251726))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 9eb506bc-aa76-4401-bb2f-7c5bef9772ab, kind: Instance { id: eab2ccae-1246-4353-8b8d-c87c0502905b }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10251726), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-3074993bd6e", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 834f6ede-be0e-4346-bf10-23744ed73332, read_only_parent: Some(Volume { block_size: 512, id: 1c2a4007-d32b-4141-8b9f-869b3243cd4f, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: f27e70fb-6921-4977-ac22-b2cf4c62b505, key: Some("kirjE5IKC4cmQ8QhZfU8V+GQKkmMB7g0qZU1CAVCOjo="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19001", "[fd00:1122:3344:101::19]:19001", "[fd00:1122:3344:101::18]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 834f6ede-be0e-4346-bf10-23744ed73332, key: Some("FJ9v4dVM4tYTRICfbqssrg12GPO7vKN1APmfBmuK2CE="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::16]:19001", "[fd00:1122:3344:101::13]:19001", "[fd00:1122:3344:101::1a]:19000"] } }] } }], boot_settings: Some(BootSettings { order: [BootOrderEntry { name: "disk-3074993bd6e" }] }), cloud_init_bytes: Some(..) }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-11-28T01:46:29.367285621Z }, propolis_addr: [fd00:1122:3344:101::1:0]:12400, migration_id: None }
32542024-11-28T01:46:29.373ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6077 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53770 req_id = 407712c7-a31b-42e8-9da2-f122f0ca1415 response_code = 200 uri = /vmms/c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32552024-11-28T01:46:29.455ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49750
32562024-11-28T01:46:29.455ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 179 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49750 req_id = 719b10ab-5c6f-43a5-b9d9-db1fd0b6d876 response_code = 200 uri = /vmms/c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4/state
32572024-11-28T01:46:29.456ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte6", ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 240, 0, 0]), slot: 0, vni: Vni { inner: 10251726 }, subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 } }) }
32582024-11-28T01:46:29.456ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
32592024-11-28T01:46:29.504ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 file = illumos-utils/src/zone.rs:325 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32602024-11-28T01:46:29.531ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 file = illumos-utils/src/zone.rs:363 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32612024-11-28T01:46:29.540ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48686
32622024-11-28T01:46:29.540ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 72 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48686 req_id = 50ed903e-f513-45f6-8f91-e24f69371694 response_code = 200 uri = /vpc-routes
32632024-11-28T01:46:29.555ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {9eb506bc-aa76-4401-bb2f-7c5bef9772ab: {10.151.1.176: {22796e78-ae6b-4653-8d60-7d04f582adb6}, 10.151.1.175: {22796e78-ae6b-4653-8d60-7d04f582adb6}}} }
32642024-11-28T01:46:30.980ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlInstance0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1:0'/> <propval type="astring" name="listen_port" value='12400'/> <propval type="astring" name="metric_addr" value='dns'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32652024-11-28T01:46:30.980ZINFOSledAgent (InstanceManager): Booting oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 zone file = illumos-utils/src/running_zone.rs:488 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 zone = oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32662024-11-28T01:46:31.234ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4") failed: Property not found. retry in 29.821915ms file = illumos-utils/src/svc.rs:36 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 zone = oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32672024-11-28T01:46:31.275ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4") failed: Property not found. retry in 77.723053ms file = illumos-utils/src/svc.rs:36 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 zone = oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32682024-11-28T01:46:31.374ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4") failed: Property not found. retry in 241.226097ms file = illumos-utils/src/svc.rs:36 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 zone = oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32692024-11-28T01:46:31.630ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4") failed: Property not found. retry in 239.80371ms file = illumos-utils/src/svc.rs:36 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 zone = oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32702024-11-28T01:46:31.889ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4") failed: Property not found. retry in 546.216053ms file = illumos-utils/src/svc.rs:36 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 zone = oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32712024-11-28T01:46:32.450ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4") failed: Property not found. retry in 705.010838ms file = illumos-utils/src/svc.rs:36 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 zone = oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32722024-11-28T01:46:33.170ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4") failed: Property not found. retry in 1.49269429s file = illumos-utils/src/svc.rs:36 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 zone = oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32732024-11-28T01:46:34.677ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4") failed: Property not found. retry in 934.048555ms file = illumos-utils/src/svc.rs:36 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 zone = oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32742024-11-28T01:46:35.625ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4") failed: Property not found. retry in 624.309297ms file = illumos-utils/src/svc.rs:36 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 zone = oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32752024-11-28T01:46:36.285ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 file = sled-agent/src/instance.rs:1786 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32762024-11-28T01:46:36.299ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4") failed: Property not found. retry in 25.591054ms file = illumos-utils/src/svc.rs:36 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32772024-11-28T01:46:36.337ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4") failed: Property not found. retry in 144.32827ms file = illumos-utils/src/svc.rs:36 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32782024-11-28T01:46:36.494ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4") failed: Property not found. retry in 282.145172ms file = illumos-utils/src/svc.rs:36 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32792024-11-28T01:46:36.630ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32802024-11-28T01:46:36.630ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32812024-11-28T01:46:36.631ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32822024-11-28T01:46:36.632ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32832024-11-28T01:46:36.632ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32842024-11-28T01:46:36.633ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32852024-11-28T01:46:36.633ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32862024-11-28T01:46:36.634ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32872024-11-28T01:46:36.634ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32882024-11-28T01:46:36.635ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32892024-11-28T01:46:36.636ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32902024-11-28T01:46:36.636ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32912024-11-28T01:46:36.637ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32922024-11-28T01:46:36.638ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32932024-11-28T01:46:36.639ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32942024-11-28T01:46:36.639ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32952024-11-28T01:46:36.641ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32962024-11-28T01:46:36.641ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
32972024-11-28T01:46:36.791ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:1795 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32982024-11-28T01:46:36.826ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:1819 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
32992024-11-28T01:46:36.966ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:932 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
33002024-11-28T01:46:36.966ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1243 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33012024-11-28T01:46:36.967ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7411237 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48686 req_id = dda3d8a8-6ffb-4e81-b065-61bacfcf2957 response_code = 204 uri = /eip-gateways
33022024-11-28T01:46:36.967ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Creating), migration_in: None, migration_out: None, time: 2024-11-28T01:46:36.967030618Z } file = sled-agent/src/instance.rs:780 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
33032024-11-28T01:46:36.967ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-11-28T01:46:36.967030618Z } propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
33042024-11-28T01:46:36.968ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-11-28T01:46:36.967030618Z }, migration_in: None, migration_out: None }
33052024-11-28T01:46:36.998ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_in: None, migration_out: None, time: 2024-11-28T01:46:36.998132683Z } file = sled-agent/src/instance.rs:780 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
33062024-11-28T01:46:36.998ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-11-28T01:46:36.998132683Z } propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
33072024-11-28T01:46:36.998ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-11-28T01:46:36.998132683Z }, migration_in: None, migration_out: None }
33082024-11-28T01:46:37.077ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 279 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48686 req_id = 2cb89965-ce4c-4816-9315-27c50354a896 response_code = 204 uri = /vpc-routes
33092024-11-28T01:46:42.188ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55764
33102024-11-28T01:46:42.188ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 393 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55764 req_id = e9293357-b439-48c2-ae41-bc47cdd651f7 response_code = 200 uri = /vmms/c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4/state
33112024-11-28T01:46:42.232ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52337
33122024-11-28T01:46:42.232ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 207 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52337 req_id = 82f4ccdd-3b7e-49b3-b18d-91910d5a5b8e response_code = 200 uri = /v2p
33132024-11-28T01:46:42.252ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41046
33142024-11-28T01:46:42.252ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 43 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41046 req_id = 0009db14-4bfc-4411-9287-b45be538dd87 response_code = 200 uri = /vpc-routes
33152024-11-28T01:46:42.263ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55643
33162024-11-28T01:46:42.263ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55643 req_id = 0c9fbb9e-6365-42d1-b677-ab57701ac068 response_code = 200 uri = /vmms/c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4/state
33172024-11-28T01:46:42.293ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {9eb506bc-aa76-4401-bb2f-7c5bef9772ab: {10.151.1.175: {22796e78-ae6b-4653-8d60-7d04f582adb6}, 10.151.1.176: {22796e78-ae6b-4653-8d60-7d04f582adb6}}} }
33182024-11-28T01:46:42.293ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:41046 req_id = 381ef6e6-cb3b-4e12-aa65-47836450fdf4 response_code = 204 uri = /eip-gateways
33192024-11-28T01:46:42.312ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 223 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:41046 req_id = 59374ef3-ca98-4bb5-959b-b8c9accd29eb response_code = 204 uri = /vpc-routes
33202024-11-28T01:46:42.338ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46257
33212024-11-28T01:46:42.339ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46257 req_id = 73e239bf-9b60-4cdf-b8f0-fc8ba2745b1b response_code = 200 uri = /vpc-routes
33222024-11-28T01:46:42.351ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47194
33232024-11-28T01:46:42.351ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
33242024-11-28T01:46:42.417ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
33252024-11-28T01:46:42.513ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 162119 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:47194 req_id = c0c91778-7cb3-45ca-a82e-bb0f208b2d80 response_code = 204 uri = /switch-ports
33262024-11-28T01:46:42.514ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65001
33272024-11-28T01:46:42.514ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59256
33282024-11-28T01:46:42.514ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {9eb506bc-aa76-4401-bb2f-7c5bef9772ab: {10.151.1.176: {22796e78-ae6b-4653-8d60-7d04f582adb6}, 10.151.1.175: {22796e78-ae6b-4653-8d60-7d04f582adb6}}} }
33292024-11-28T01:46:42.514ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 313 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:46257 req_id = 42cf053d-cfd9-4b27-996a-ad36c77a5b9f response_code = 204 uri = /eip-gateways
33302024-11-28T01:46:42.514ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39255
33312024-11-28T01:46:42.514ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 352 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47194 req_id = ddae8245-a394-40e4-aaee-84bc8df1ca38 response_code = 200 uri = /network-bootstore-config
33322024-11-28T01:46:42.514ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 205 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65001 req_id = 54281520-abd4-469a-be18-39252e66128b response_code = 200 uri = /vpc-routes
33332024-11-28T01:46:42.514ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 215 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59256 req_id = f669f847-40e9-458b-8eea-45f69d2ac8ce response_code = 200 uri = /vmms/c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4/state
33342024-11-28T01:46:42.514ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
33352024-11-28T01:46:42.548ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1220 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:46257 req_id = 4b0c706c-2570-4244-97d4-78a1c15415a5 response_code = 204 uri = /vpc-routes
33362024-11-28T01:46:42.567ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {9eb506bc-aa76-4401-bb2f-7c5bef9772ab: {10.151.1.175: {22796e78-ae6b-4653-8d60-7d04f582adb6}, 10.151.1.176: {22796e78-ae6b-4653-8d60-7d04f582adb6}}} }
33372024-11-28T01:46:42.568ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 765 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:65001 req_id = 32000629-c48a-4a1d-88b0-20c0875fb3c2 response_code = 204 uri = /eip-gateways
33382024-11-28T01:46:42.581ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
33392024-11-28T01:46:42.590ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 586 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:65001 req_id = 23a3a634-7afe-493b-96f8-0a7167e210da response_code = 204 uri = /vpc-routes
33402024-11-28T01:46:42.626ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62844
33412024-11-28T01:46:42.627ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 249 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62844 req_id = d2ce812c-c0a5-49f3-b170-3508fa470354 response_code = 200 uri = /v2p
33422024-11-28T01:46:42.669ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 155003 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:39255 req_id = e056f5f2-3f4f-47b3-bfcf-95e2107a679a response_code = 204 uri = /switch-ports
33432024-11-28T01:46:42.670ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 192 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39255 req_id = f65ebfc7-b7a7-4cbd-b0d9-c530b6649755 response_code = 200 uri = /network-bootstore-config
33442024-11-28T01:46:42.686ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43909
33452024-11-28T01:46:42.686ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43909 req_id = 754e4a70-9597-4a29-b012-9baa78256227 response_code = 200 uri = /v2p
33462024-11-28T01:46:42.724ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63458
33472024-11-28T01:46:42.724ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
33482024-11-28T01:46:42.773ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
33492024-11-28T01:46:42.846ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 122266 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:63458 req_id = 71c86592-6b6e-4806-9abc-d4d873cfb6b2 response_code = 204 uri = /switch-ports
33502024-11-28T01:46:42.847ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63458 req_id = 010e0379-5cf1-4dbb-98b9-9f6654332917 response_code = 200 uri = /network-bootstore-config
33512024-11-28T01:46:46.634ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33522024-11-28T01:46:46.642ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33532024-11-28T01:46:46.643ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33542024-11-28T01:46:46.644ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33552024-11-28T01:46:46.644ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33562024-11-28T01:46:46.645ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33572024-11-28T01:46:46.646ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33582024-11-28T01:46:46.646ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33592024-11-28T01:46:46.647ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33602024-11-28T01:46:46.648ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33612024-11-28T01:46:46.648ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33622024-11-28T01:46:46.649ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33632024-11-28T01:46:46.650ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33642024-11-28T01:46:46.651ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33652024-11-28T01:46:46.651ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33662024-11-28T01:46:46.652ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33672024-11-28T01:46:46.653ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33682024-11-28T01:46:46.654ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33692024-11-28T01:46:46.929ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53691
33702024-11-28T01:46:46.929ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 382 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53691 req_id = 13b43c62-1374-4306-9ba8-232e2de21f28 response_code = 200 uri = /vmms/c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4/state
33712024-11-28T01:46:46.930ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_in: None, migration_out: None, time: 2024-11-28T01:46:46.930041514Z } file = sled-agent/src/instance.rs:780 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
33722024-11-28T01:46:46.930ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-11-28T01:46:46.930041514Z } propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
33732024-11-28T01:46:46.930ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 state = SledVmmState { vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-11-28T01:46:46.930041514Z }, migration_in: None, migration_out: None }
33742024-11-28T01:46:46.965ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_in: None, migration_out: None, time: 2024-11-28T01:46:46.965716384Z } file = sled-agent/src/instance.rs:780 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
33752024-11-28T01:46:46.965ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-11-28T01:46:46.965716384Z } propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
33762024-11-28T01:46:46.965ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:816 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
33772024-11-28T01:46:46.965ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
33782024-11-28T01:46:46.965ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:602 id = 14423359006634714998 n_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33792024-11-28T01:46:46.965ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:602 id = 8244727884455237683 n_samples = 6 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
33802024-11-28T01:46:46.966ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
33812024-11-28T01:46:46.966ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/2d6a91dc-89a8-488e-9efc-6f5bec9b10d5/debug/bundle/zone", "/pool/int/7f63c436-f542-43a4-9af6-de72a4c8391f/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/a6e234b1-bf4c-4b91-92ee-604781b835ee/crypt/debug", "/pool/ext/8164ba35-0eff-436a-9b6b-80912419e510/crypt/debug", "/pool/ext/0176efd8-6a11-46a9-8bcd-a97e447fcb50/crypt/debug", "/pool/ext/beaaa347-4478-48b4-b888-a5873241ded0/crypt/debug", "/pool/ext/216077bb-f8f0-4fbf-8aa0-1db2249e65da/crypt/debug", "/pool/ext/977ab5dc-c743-4525-b172-ecf8a9034abe/crypt/debug", "/pool/ext/2fcc59ee-335a-4c80-bbcf-e66263237ec3/crypt/debug", "/pool/ext/c7ca3a79-57f5-4e47-8807-03bd67966814/crypt/debug", "/pool/ext/75598bf6-d28b-4334-94db-398fc2a8b539/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:339 zone_name = oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
33822024-11-28T01:46:47.667ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1142 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4", bundle_id: 6dd9e678-ce46-4323-a65e-c91651d78c71 }, time_created: 2024-11-28T01:46:46.967496664Z, version: 0, cause: TerminatedInstance }
33832024-11-28T01:46:47.667ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 file = sled-agent/src/instance.rs:1032 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
33842024-11-28T01:46:49.467ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:279 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
33852024-11-28T01:46:49.483ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:1000 id = RouterId { vni: Vni(10251726), kind: System }
33862024-11-28T01:46:49.483ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:1000 id = RouterId { vni: Vni(10251726), kind: Custom(V4(Ipv4Net { addr: 172.30.0.0, width: 22 })) }
33872024-11-28T01:46:49.484ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-28T01:46:49.483981186Z }, migration_in: None, migration_out: None }
33882024-11-28T01:46:49.509ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-28T01:46:49.483981186Z }, migration_in: None, migration_out: None }
33892024-11-28T01:46:49.512ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:969 instance_id = eab2ccae-1246-4353-8b8d-c87c0502905b propolis_id = c0ea32fd-8b53-4ed4-afa0-2ecbfc48bee4
33902024-11-28T01:46:49.925ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34679
33912024-11-28T01:46:49.925ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34679 req_id = 32eacb0d-8f42-43ae-a599-4bfa452e9f82 response_code = 200 uri = /vpc-routes
33922024-11-28T01:46:49.968ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {9eb506bc-aa76-4401-bb2f-7c5bef9772ab: {10.151.1.176: {22796e78-ae6b-4653-8d60-7d04f582adb6}, 10.151.1.175: {22796e78-ae6b-4653-8d60-7d04f582adb6}}} }
33932024-11-28T01:46:49.968ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34679 req_id = ff1fd1e7-6f04-404e-a1db-94311f593ce3 response_code = 204 uri = /eip-gateways
33942024-11-28T01:46:49.977ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 216 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34679 req_id = 3c9abf73-8d40-47f5-8a87-5bc7454ddfaf response_code = 204 uri = /vpc-routes
33952024-11-28T01:46:50.059ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40983
33962024-11-28T01:46:50.059ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40983 req_id = 74dc65d2-0adc-483d-a709-e4cb176200a7 response_code = 200 uri = /v2p
33972024-11-28T01:46:50.246ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53731
33982024-11-28T01:46:50.246ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 157 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53731 req_id = dba0d3aa-945c-4d33-aacd-1e5347826f57 response_code = 200 uri = /v2p
33992024-11-28T01:46:50.409ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54407
34002024-11-28T01:46:50.409ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54407 req_id = cbb25504-bb21-45db-ac43-d2c0b60504ab response_code = 200 uri = /vpc-routes
34012024-11-28T01:46:50.423ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
34022024-11-28T01:46:50.423ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1243 sled_id = 12368a57-a643-445a-b32d-da10700bd20e
34032024-11-28T01:46:50.423ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54407 req_id = 7215fb03-2773-4985-b3ed-3031d957f8f6 response_code = 204 uri = /eip-gateways
34042024-11-28T01:46:50.431ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 238 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54407 req_id = 82d0e2da-1e28-498c-a1d8-97d6f8396a29 response_code = 204 uri = /vpc-routes
34052024-11-28T01:46:51.103ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54430
34062024-11-28T01:46:51.103ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54430 req_id = 584487a4-aab6-4c6a-8b0d-1e5bfacd2df8 response_code = 200 uri = /vpc-routes
34072024-11-28T01:46:51.126ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1217 sled_id = 12368a57-a643-445a-b32d-da10700bd20e values = ExternalIpGatewayMap { mappings: {} }
34082024-11-28T01:46:51.126ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54430 req_id = 27a06cf2-0f15-4c07-a9e3-fa12715554c3 response_code = 204 uri = /eip-gateways
34092024-11-28T01:46:51.137ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 261 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54430 req_id = 8e8bbccd-53d2-475f-a85c-a0181ee03a25 response_code = 204 uri = /vpc-routes