Max level shown:
1 [ Nov 13 05:06:01 Enabled. ]
2 [ Nov 13 05:06:01 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Nov 13 05:06:01 Method "start" exited with status 0. ]
4 [ Nov 13 05:06:01 Rereading configuration. ]
5 [ Nov 13 05:06:01 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72024-11-13T05:06:01.721ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:113
82024-11-13T05:06:01.721ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:255
92024-11-13T05:06:01.774ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:262
102024-11-13T05:06:01.775ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
112024-11-13T05:06:01.889ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
122024-11-13T05:06:01.898ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
132024-11-13T05:06:01.906ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:272 path = /var/run/oxide/
142024-11-13T05:06:02.509ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fdb0:18c0:4d0d:a02a::, width: 64 }
152024-11-13T05:06:02.646ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:126
162024-11-13T05:06:02.646ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:138
172024-11-13T05:06:02.646ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:151
182024-11-13T05:06:02.647ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:173 nongimlet_observed_disks = [] sled_mode = Scrimlet { asic: SoftNpuZone }
192024-11-13T05:06:02.647ZINFOSledAgent (DumpSetup-worker): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:520
202024-11-13T05:06:02.647ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:717
212024-11-13T05:06:02.650ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "sock", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:624
222024-11-13T05:06:02.650ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:766
232024-11-13T05:06:02.650ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:187
242024-11-13T05:06:02.650ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:825
252024-11-13T05:06:02.650ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_0.vdev
262024-11-13T05:06:02.650ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = None
272024-11-13T05:06:02.650ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
282024-11-13T05:06:02.650ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
292024-11-13T05:06:02.650ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } path = /scratch/m2_0.vdev
302024-11-13T05:06:02.661ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_0.vdev file = sled-hardware/src/disk.rs:367
312024-11-13T05:06:02.661ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = 3c73febf-14eb-49a3-9af1-023b1a81de43 (zpool)
322024-11-13T05:06:02.780ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 3c73febf-14eb-49a3-9af1-023b1a81de43 (zpool), kind: Internal }
332024-11-13T05:06:03.356ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:603
342024-11-13T05:06:03.356ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_1.vdev
352024-11-13T05:06:03.356ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
362024-11-13T05:06:03.356ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
372024-11-13T05:06:03.356ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
382024-11-13T05:06:03.356ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
392024-11-13T05:06:03.356ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } path = /scratch/m2_1.vdev
402024-11-13T05:06:03.365ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_1.vdev file = sled-hardware/src/disk.rs:367
412024-11-13T05:06:03.365ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = 147a026f-d4f7-4267-9015-fa2814ab1bfc (zpool)
422024-11-13T05:06:03.470ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 147a026f-d4f7-4267-9015-fa2814ab1bfc (zpool), kind: Internal }
432024-11-13T05:06:04.023ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_0.vdev
442024-11-13T05:06:04.023ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 3c73febf-14eb-49a3-9af1-023b1a81de43 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 147a026f-d4f7-4267-9015-fa2814ab1bfc (zpool), kind: Internal } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
452024-11-13T05:06:04.023ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
462024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
472024-11-13T05:06:04.023ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
482024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
492024-11-13T05:06:04.023ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_1.vdev
502024-11-13T05:06:04.023ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 3c73febf-14eb-49a3-9af1-023b1a81de43 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 147a026f-d4f7-4267-9015-fa2814ab1bfc (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
512024-11-13T05:06:04.023ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
522024-11-13T05:06:04.023ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
532024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
542024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
552024-11-13T05:06:04.023ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_2.vdev
562024-11-13T05:06:04.023ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 3c73febf-14eb-49a3-9af1-023b1a81de43 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 147a026f-d4f7-4267-9015-fa2814ab1bfc (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
572024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
582024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
592024-11-13T05:06:04.023ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
602024-11-13T05:06:04.023ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
612024-11-13T05:06:04.023ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_3.vdev
622024-11-13T05:06:04.023ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 3c73febf-14eb-49a3-9af1-023b1a81de43 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 147a026f-d4f7-4267-9015-fa2814ab1bfc (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
632024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
642024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
652024-11-13T05:06:04.023ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
662024-11-13T05:06:04.023ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
672024-11-13T05:06:04.023ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_4.vdev
682024-11-13T05:06:04.023ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 3c73febf-14eb-49a3-9af1-023b1a81de43 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 147a026f-d4f7-4267-9015-fa2814ab1bfc (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
692024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
702024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
712024-11-13T05:06:04.023ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
722024-11-13T05:06:04.023ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
732024-11-13T05:06:04.023ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_5.vdev
742024-11-13T05:06:04.023ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 3c73febf-14eb-49a3-9af1-023b1a81de43 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 147a026f-d4f7-4267-9015-fa2814ab1bfc (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
752024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
762024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
772024-11-13T05:06:04.023ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
782024-11-13T05:06:04.023ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
792024-11-13T05:06:04.023ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_6.vdev
802024-11-13T05:06:04.023ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 3c73febf-14eb-49a3-9af1-023b1a81de43 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 147a026f-d4f7-4267-9015-fa2814ab1bfc (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
812024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
822024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
832024-11-13T05:06:04.023ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
842024-11-13T05:06:04.023ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
852024-11-13T05:06:04.023ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_7.vdev
862024-11-13T05:06:04.023ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 3c73febf-14eb-49a3-9af1-023b1a81de43 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 147a026f-d4f7-4267-9015-fa2814ab1bfc (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
872024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
882024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
892024-11-13T05:06:04.023ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
902024-11-13T05:06:04.023ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
912024-11-13T05:06:04.023ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_8.vdev
922024-11-13T05:06:04.023ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
932024-11-13T05:06:04.023ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
942024-11-13T05:06:04.023ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 3c73febf-14eb-49a3-9af1-023b1a81de43 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 147a026f-d4f7-4267-9015-fa2814ab1bfc (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
952024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
962024-11-13T05:06:04.023ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:97
972024-11-13T05:06:04.023ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
982024-11-13T05:06:04.023ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-agent/src/long_running_tasks.rs:99
992024-11-13T05:06:04.023ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1002024-11-13T05:06:04.024ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 3c73febf-14eb-49a3-9af1-023b1a81de43 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 147a026f-d4f7-4267-9015-fa2814ab1bfc (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
1012024-11-13T05:06:04.024ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
1022024-11-13T05:06:04.024ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1032024-11-13T05:06:04.024ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:211
1042024-11-13T05:06:04.024ZINFOSledAgent (bootstore): No ledger in /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1052024-11-13T05:06:04.024ZINFOSledAgent (bootstore): No ledger in /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1062024-11-13T05:06:04.024ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-sock-i86pc
1072024-11-13T05:06:04.024ZINFOSledAgent (bootstore): No ledger in /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1082024-11-13T05:06:04.024ZINFOSledAgent (bootstore): No ledger in /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1092024-11-13T05:06:04.024ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-sock-i86pc
1102024-11-13T05:06:04.024ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:216
1112024-11-13T05:06:04.024ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:231
1122024-11-13T05:06:04.035ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:736
1132024-11-13T05:06:04.035ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 5344, tv_nsec: 753763380 } time_to_next_cleanup = 599.99993998s
1142024-11-13T05:06:04.035ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3947
1152024-11-13T05:06:04.035ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1162024-11-13T05:06:04.036ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/sled-agent-request.json file = common/src/ledger.rs:188
1172024-11-13T05:06:04.038ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/sled-agent-request.json file = common/src/ledger.rs:188
1182024-11-13T05:06:04.039ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:197 local_addr = [fdb0:18c0:4d0d:a02a::1]:80
1192024-11-13T05:06:04.041ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:51 local_addr = [fdb0:18c0:4d0d:a02a::1]:12346
1202024-11-13T05:06:04.041ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, rib_priority: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, tx_eq: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, } file = sled-agent/src/rack_setup/service.rs:1198
1212024-11-13T05:06:04.041ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1222024-11-13T05:06:04.041ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1232024-11-13T05:06:04.041ZINFOSledAgent (RSS): No ledger in /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1242024-11-13T05:06:04.041ZINFOSledAgent (RSS): No ledger in /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1252024-11-13T05:06:04.041ZINFOSledAgent (RSS): RSS configuration has not been fully applied yet file = sled-agent/src/rack_setup/service.rs:1261
1262024-11-13T05:06:04.041ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1272024-11-13T05:06:04.041ZINFOSledAgent (RSS): No ledger in /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/rss-sled-plan.json file = common/src/ledger.rs:188
1282024-11-13T05:06:04.041ZINFOSledAgent (RSS): No ledger in /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/rss-sled-plan.json file = common/src/ledger.rs:188
1292024-11-13T05:06:04.041ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:1303
1302024-11-13T05:06:04.041ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:156
1312024-11-13T05:06:04.041ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:a02a::1 file = sled-agent/src/rack_setup/plan/sled.rs:129
1322024-11-13T05:06:04.041ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1332024-11-13T05:06:04.041ZINFOSledAgent (RSS): Writing ledger to /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1342024-11-13T05:06:04.042ZINFOSledAgent (RSS): Writing ledger to /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/.rss-sled-plan.json.tmp file = common/src/ledger.rs:199
1352024-11-13T05:06:04.042ZINFOSledAgent (RSS): Sled plan written to storage: Plan { rack_id: b7b41d81-f310-4949-894b-9f77b62a6c9b, sleds: { [fdb0:18c0:4d0d:a02a::1]:12346: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 4d997277-f640-48d7-af56-6df261d5afe1 (sled), rack_id: b7b41d81-f310-4949-894b-9f77b62a6c9b, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64, }, }, }, }, }, config: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, rib_priority: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, tx_eq: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, }, } file = sled-agent/src/rack_setup/plan/sled.rs:176
1362024-11-13T05:06:04.042ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:1337
1372024-11-13T05:06:04.042ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
1382024-11-13T05:06:04.042ZINFOSledAgent (bootstore): Writing ledger to /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1392024-11-13T05:06:04.042ZINFOSledAgent (bootstore): Writing ledger to /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1402024-11-13T05:06:04.042ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
1412024-11-13T05:06:04.042ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:232 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 4d997277-f640-48d7-af56-6df261d5afe1 (sled), rack_id: b7b41d81-f310-4949-894b-9f77b62a6c9b, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
1422024-11-13T05:06:04.043ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:4244
1432024-11-13T05:06:04.043ZINFOSledAgent (ServiceManager): Starting switch zone file = sled-agent/src/services.rs:4644
1442024-11-13T05:06:04.047ZINFOSledAgent (SledAgentSprocketsClient): Loaded keys and certs BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/client.rs:50
1452024-11-13T05:06:04.052ZINFOSledAgent (SledAgentSprocketsClient): Certificate chain verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1462024-11-13T05:06:04.052ZINFOSledAgent (SledAgentSprocketsClient): Signature verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1472024-11-13T05:06:04.057ZINFOSledAgent (SledAgentSprocketsServer): Certificate chain verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1482024-11-13T05:06:04.057ZINFOSledAgent (SledAgentSprocketsServer): Signature verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1492024-11-13T05:06:04.057ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:74 remote_addr = [fdb0:18c0:4d0d:a02a::1]:60526
1502024-11-13T05:06:04.057ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 4d997277-f640-48d7-af56-6df261d5afe1 (sled), rack_id: b7b41d81-f310-4949-894b-9f77b62a6c9b, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } file = sled-agent/src/bootstrap/server.rs:362
1512024-11-13T05:06:04.057ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:377
1522024-11-13T05:06:04.057ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:44
1532024-11-13T05:06:04.057ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1542024-11-13T05:06:04.057ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:603
1552024-11-13T05:06:04.057ZINFOSledAgent (StorageManager): Loading storage config file = sled-storage/src/manager.rs:809
1562024-11-13T05:06:04.057ZINFOSledAgent (StorageManager): No ledger in /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1572024-11-13T05:06:04.057ZINFOSledAgent (StorageManager): No ledger in /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1582024-11-13T05:06:04.057ZINFOSledAgent (StorageManager): No ledger of physical disks exists file = sled-storage/src/manager.rs:798
1592024-11-13T05:06:04.057ZINFOSledAgent (StorageManager): KeyManager ready, but no ledger detected file = sled-storage/src/manager.rs:826
1602024-11-13T05:06:04.057ZINFOSledAgent (StorageManager): automatically managing disks file = sled-storage/src/manager.rs:592
1612024-11-13T05:06:04.057ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:101::, width: 64 }
1622024-11-13T05:06:04.057ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
1632024-11-13T05:06:04.057ZINFOSledAgent (StorageManager): Successfully synchronized disks without error file = sled-storage/src/manager.rs:697 result = DisksManagementResult { status: [] }
1642024-11-13T05:06:04.075ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1652024-11-13T05:06:04.096ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:325
1662024-11-13T05:06:04.108ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:408 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
1672024-11-13T05:06:04.108ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1682024-11-13T05:06:04.108ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:420 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
1692024-11-13T05:06:04.108ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1702024-11-13T05:06:04.111ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 81.271985ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "7102f1e3-bfcd-4921-9090-723cdbc6bd5d", "content-length": "124", "date": "Wed, 13 Nov 2024 05:06:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "7102f1e3-bfcd-4921-9090-723cdbc6bd5d" } file = clients/ddm-admin-client/src/lib.rs:177
1712024-11-13T05:06:04.113ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:363
1722024-11-13T05:06:04.121ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_3c73febf-14eb-49a3-9af1-023b1a81de43/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1732024-11-13T05:06:04.147ZINFOSledAgent: successfully created encrypted zvol: name="oxi_3c73febf-14eb-49a3-9af1-023b1a81de43/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1742024-11-13T05:06:04.147ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_3c73febf-14eb-49a3-9af1-023b1a81de43/swap" file = sled-agent/src/swap_device.rs:96
1752024-11-13T05:06:04.151ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:435 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
1762024-11-13T05:06:04.152ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:128
1772024-11-13T05:06:04.152ZINFOSledAgent (BackingFs): Ensuring dataset oxi_3c73febf-14eb-49a3-9af1-023b1a81de43/backing/fmd file = sled-agent/src/backing_fs.rs:138
1782024-11-13T05:06:04.193ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1792024-11-13T05:06:04.193ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 143.263507ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "6212f048-9786-41cb-8f69-d559aaf535c1", "content-length": "124", "date": "Wed, 13 Nov 2024 05:06:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "6212f048-9786-41cb-8f69-d559aaf535c1" } file = clients/ddm-admin-client/src/lib.rs:177
1802024-11-13T05:06:04.249ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:173
1812024-11-13T05:06:04.337ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1822024-11-13T05:06:04.338ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 131.223367ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "cf54de48-9e72-4366-b0b1-89f0e080c7f8", "content-length": "124", "date": "Wed, 13 Nov 2024 05:06:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "cf54de48-9e72-4366-b0b1-89f0e080c7f8" } file = clients/ddm-admin-client/src/lib.rs:177
1832024-11-13T05:06:04.467ZINFOSledAgent (BackingFs): Mounting oxi_3c73febf-14eb-49a3-9af1-023b1a81de43/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:181
1842024-11-13T05:06:04.471ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1852024-11-13T05:06:04.471ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 258.497637ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "5cd25499-a8bb-4e43-8572-c6351de7085a", "content-length": "124", "date": "Wed, 13 Nov 2024 05:06:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "5cd25499-a8bb-4e43-8572-c6351de7085a" } file = clients/ddm-admin-client/src/lib.rs:177
1862024-11-13T05:06:04.479ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:189
1872024-11-13T05:06:04.480ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:189
1882024-11-13T05:06:04.480ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:189
1892024-11-13T05:06:04.480ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:195
1902024-11-13T05:06:04.732ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1912024-11-13T05:06:04.733ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 239.901957ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "37bacc2c-a489-45f9-af23-b73a380ffb94", "content-length": "124", "date": "Wed, 13 Nov 2024 05:06:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "37bacc2c-a489-45f9-af23-b73a380ffb94" } file = clients/ddm-admin-client/src/lib.rs:177
1922024-11-13T05:06:04.974ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1932024-11-13T05:06:04.975ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 269.249742ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "09b75ddf-ac15-48aa-9e78-bfad5868a326", "content-length": "124", "date": "Wed, 13 Nov 2024 05:06:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "09b75ddf-ac15-48aa-9e78-bfad5868a326" } file = clients/ddm-admin-client/src/lib.rs:177
1942024-11-13T05:06:05.219ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:93 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
1952024-11-13T05:06:05.225ZINFOSledAgent (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:197 local_addr = [fd00:1122:3344:101::1]:38057 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
1962024-11-13T05:06:05.225ZINFOSledAgent (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
1972024-11-13T05:06:05.225ZINFOSledAgent (producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::1]:38057 file = oximeter/producer/src/lib.rs:283 interval = 30s producer_id = 4d997277-f640-48d7-af56-6df261d5afe1 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
1982024-11-13T05:06:05.226ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 194.516588ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
1992024-11-13T05:06:05.227ZINFOSledAgent (VmmReservoirManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/vmm_reservoir.rs:267 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
2002024-11-13T05:06:05.227ZINFOSledAgent (VmmReservoirManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:275 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
2012024-11-13T05:06:05.227ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:920 underlay_address = fd00:1122:3344:101::1
2022024-11-13T05:06:05.227ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
2032024-11-13T05:06:05.245ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
2042024-11-13T05:06:05.423ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 726.679801ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
2052024-11-13T05:06:06.150ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 769.484967ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
2062024-11-13T05:06:06.922ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 2.685727353s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
2072024-11-13T05:06:07.743ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2668
2082024-11-13T05:06:07.743ZINFOSledAgent (ServiceManager): Setting up lldpd service file = sled-agent/src/services.rs:2913
2092024-11-13T05:06:07.743ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2538
2102024-11-13T05:06:07.743ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2585
2112024-11-13T05:06:07.743ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:2994
2122024-11-13T05:06:07.743ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:3039
2132024-11-13T05:06:07.744ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2838
2142024-11-13T05:06:07.746ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2579
2152024-11-13T05:06:07.746ZINFOSledAgent (ServiceManager): Profile for oxz_switch: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/mgs"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12225'/> <propval type="astring" name="id" value='eb1311f8-6807-471a-a8f7-8fe643dea75d'/> <propval type="astring" name="rack_id" value='b7b41d81-f310-4949-894b-9f77b62a6c9b'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/wicketd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12226'/> <propval type="astring" name="artifact-address" value='[fdb0:18c0:4d0d:a02a::2]:12227'/> <propval type="astring" name="baseboard-file" value='/opt/oxide/baseboard.json'/> <propval type="astring" name="mgs-address" value='[::1]:12225'/> <propval type="astring" name="nexus-proxy-address" value='[::]:12229'/> <propval type="astring" name="rack-subnet" value='fd00:1122:3344::'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/switch_zone_setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="gz_local_link_addr" value='fe80::8:20ff:fef3:19cf'/> <property type="astring" name="link_local_links"> <astring_list> <value_node value="tfportrear0_0" /> <value_node value="oxBootstrap0" /> </astring_list> </property> <propval type="astring" name="bootstrap_addr" value='fdb0:18c0:4d0d:a02a::2'/> <propval type="astring" name="bootstrap_vnic" value='oxBootstrap0'/> <propval type="astring" name="baseboard_info" value='{"type":"pc","identifier":"sock","model":"i86pc"}'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/dendrite"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="rack_id" value='b7b41d81-f310-4949-894b-9f77b62a6c9b'/> <propval type="astring" name="sled_id" value='4d997277-f640-48d7-af56-6df261d5afe1'/> <propval type="astring" name="sled_model" value='i86pc'/> <propval type="astring" name="sled_serial" value='sock'/> <propval type="astring" name="sled_revision" value='0'/> <propval type="astring" name="address" value='[::1]:12224'/> <propval type="astring" name="front_ports" value='1'/> <propval type="astring" name="rear_ports" value='1'/> <propval type="astring" name="port_config" value='/opt/oxide/dendrite/misc/softnpu_single_sled_config.toml'/> <propval type="astring" name="mgmt" value='uds'/> <propval type="astring" name="uds_path" value='/opt/softnpu/stuff'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/tfport"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="host" value='[::1]'/> <propval type="astring" name="port" value='12224'/> <propval type="astring" name="flags" value='--sync-only'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/lldpd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="board_rev" value='softnpu_front_1_rear_1'/> <propval type="astring" name="scrimlet_id" value='sock'/> <propval type="astring" name="scrimlet_model" value='i86pc'/> <propval type="astring" name="address" value='[::1]:12230'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/pumpkind"> </service> <service version="1" type="service" name="oxide/mgd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_uuid" value='4d997277-f640-48d7-af56-6df261d5afe1'/> <propval type="astring" name="rack_uuid" value='b7b41d81-f310-4949-894b-9f77b62a6c9b'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/mg-ddm"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="mode" value='transit'/> <propval type="astring" name="dendrite" value='true'/> <propval type="astring" name="sled_uuid" value='4d997277-f640-48d7-af56-6df261d5afe1'/> <propval type="astring" name="rack_uuid" value='b7b41d81-f310-4949-894b-9f77b62a6c9b'/> <propval type="astring" name="interfaces" value='tfportrear0_0/ll'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/uplink"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
2162024-11-13T05:06:07.746ZINFOSledAgent (ServiceManager): Booting oxz_switch zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_switch
2172024-11-13T05:06:08.150ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 45.766286ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2182024-11-13T05:06:08.353ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 81.855437ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2192024-11-13T05:06:08.545ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 143.538205ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2202024-11-13T05:06:08.740ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 585.778952ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2212024-11-13T05:06:09.353ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.073060115s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2222024-11-13T05:06:09.610ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 3.106326576s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
2232024-11-13T05:06:10.611ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 634.596833ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2242024-11-13T05:06:11.301ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 685.213668ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2252024-11-13T05:06:12.127ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 543.619745ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2262024-11-13T05:06:12.766ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 813.005418ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2272024-11-13T05:06:12.767ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 10.397838813s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
2282024-11-13T05:06:13.918ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/4d997277-f640-48d7-af56-6df261d5afe1) file = sled-agent/src/nexus.rs:527 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
2292024-11-13T05:06:13.947ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2302024-11-13T05:06:13.948ZINFOSledAgent (dropshot (Repo Depot)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:197 local_addr = [fd00:1122:3344:101::1]:12348 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
2312024-11-13T05:06:13.949ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:197 local_addr = [fd00:1122:3344:101::1]:12345
2322024-11-13T05:06:13.949ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:432
2332024-11-13T05:06:13.949ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2342024-11-13T05:06:13.949ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2352024-11-13T05:06:13.949ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2362024-11-13T05:06:13.949ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:138
2372024-11-13T05:06:13.949ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56 }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [PortConfigV2 { routes: [RouteConfig { destination: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), nexthop: 10.151.2.100, vlan_id: None, rib_priority: None }], addresses: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: None, bgp_peers: [], autoneg: false, lldp: None, tx_eq: None }], bgp: [], bfd: [] })))
2382024-11-13T05:06:13.949ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3947
2392024-11-13T05:06:13.949ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:81 remote_addr = [fdb0:18c0:4d0d:a02a::1]:60526
2402024-11-13T05:06:14.127ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:117 peer = [fdb0:18c0:4d0d:a02a::1]:12346
2412024-11-13T05:06:14.127ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:252 target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
2422024-11-13T05:06:14.128ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2432024-11-13T05:06:14.128ZINFOSledAgent (RSS): No ledger in /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/rss-service-plan-v5.json file = common/src/ledger.rs:188
2442024-11-13T05:06:14.128ZINFOSledAgent (RSS): No ledger in /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/rss-service-plan-v5.json file = common/src/ledger.rs:188
2452024-11-13T05:06:14.128ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2462024-11-13T05:06:14.128ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2472024-11-13T05:06:14.128ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2482024-11-13T05:06:14.128ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2492024-11-13T05:06:14.133ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:4268 new = [fd00:1122:3344:101::2, ::1] old = [::1]
2502024-11-13T05:06:14.133ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists file = sled-agent/src/services.rs:4284
2512024-11-13T05:06:14.133ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::2, prefix: 64 })) file = illumos-utils/src/running_zone.rs:534 zone = oxz_switch
2522024-11-13T05:06:14.373ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:39685
2532024-11-13T05:06:14.374ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2542024-11-13T05:06:14.374ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2552024-11-13T05:06:14.374ZINFOSledAgent (ServiceManager): No ledger in /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/omicron-zones.json file = common/src/ledger.rs:188
2562024-11-13T05:06:14.374ZINFOSledAgent (ServiceManager): No ledger in /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/omicron-zones.json file = common/src/ledger.rs:188
2572024-11-13T05:06:14.374ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 417 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:39685 req_id = 35d019d4-8dd3-45da-9063-1bc5d4a6a6f1 response_code = 200 uri = /inventory
2582024-11-13T05:06:14.404ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:51868
2592024-11-13T05:06:14.404ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:51868 req_id = ab16217c-1b66-4e7d-9da4-173e41dba7c6 response_code = 200 uri = /sled-role
2602024-11-13T05:06:14.404ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2612024-11-13T05:06:14.404ZINFOSledAgent (RSS): Writing ledger to /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/.rss-service-plan-v5.json.tmp file = common/src/ledger.rs:199
2622024-11-13T05:06:14.405ZINFOSledAgent (RSS): Writing ledger to /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/.rss-service-plan-v5.json.tmp file = common/src/ledger.rs:199
2632024-11-13T05:06:14.405ZINFOSledAgent (RSS): Service plan written to storage file = sled-agent/src/rack_setup/plan/service.rs:980
2642024-11-13T05:06:14.434ZINFOSledAgent (RSS): attempting to set up sled's storage: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 60476949-5bd4-4ee6-a40c-5fade70cf988, pool_id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: d25770c4-14b4-4185-a608-f89f7df6db28, pool_id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 43980c70-e868-4500-b4f7-a69dcc27cbcb, pool_id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 636c893c-2cd2-40d8-a4eb-d2443cbcfbb1, pool_id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: a7213634-e4b4-4e53-91c4-309f81e53c86, pool_id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 6702b36d-bf3e-4c7f-8792-91c63e562b20, pool_id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 44dbfb3e-58b2-454a-bcfa-1df5f229fe62, pool_id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: 132569ab-8e20-459d-a70d-3c642397cc1a, pool_id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: d4479531-4ca2-4b33-9318-c4b5eaff0b97, pool_id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool) }] } file = sled-agent/src/rack_setup/service.rs:382 sled_address = [fd00:1122:3344:101::1]:12345
2652024-11-13T05:06:14.435ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:36484
2662024-11-13T05:06:14.435ZINFOSledAgent: physical disks ensure file = sled-agent/src/sled_agent.rs:884 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
2672024-11-13T05:06:14.435ZINFOSledAgent (StorageManager): Received OmicronPhysicalDisksEnsure { config: OmicronPhysicalDisksConfig { generation: Generation(1), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 60476949-5bd4-4ee6-a40c-5fade70cf988, pool_id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: d25770c4-14b4-4185-a608-f89f7df6db28, pool_id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 43980c70-e868-4500-b4f7-a69dcc27cbcb, pool_id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 636c893c-2cd2-40d8-a4eb-d2443cbcfbb1, pool_id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: a7213634-e4b4-4e53-91c4-309f81e53c86, pool_id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 6702b36d-bf3e-4c7f-8792-91c63e562b20, pool_id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 44dbfb3e-58b2-454a-bcfa-1df5f229fe62, pool_id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: 132569ab-8e20-459d-a70d-3c642397cc1a, pool_id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: d4479531-4ca2-4b33-9318-c4b5eaff0b97, pool_id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool) }] }, tx: ... } file = sled-storage/src/manager.rs:603
2682024-11-13T05:06:14.435ZINFOSledAgent (StorageManager): No ledger in /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2692024-11-13T05:06:14.435ZINFOSledAgent (StorageManager): No ledger in /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2702024-11-13T05:06:14.435ZINFOSledAgent (StorageManager): No previously-stored 'requested disks', creating new ledger file = sled-storage/src/manager.rs:1171 request = omicron_physical_disks_ensure
2712024-11-13T05:06:14.435ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
2722024-11-13T05:06:14.435ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:342
2732024-11-13T05:06:14.435ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2742024-11-13T05:06:14.435ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } path = /scratch/u2_0.vdev
2752024-11-13T05:06:14.592ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_0.vdev file = sled-hardware/src/disk.rs:367
2762024-11-13T05:06:14.592ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool)
2772024-11-13T05:06:15.029ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/4d997277-f640-48d7-af56-6df261d5afe1) file = sled-agent/src/probe_manager.rs:231 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
2782024-11-13T05:06:15.243ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 11661085320902503456 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
2792024-11-13T05:06:15.243ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 2533699652135011376 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
2802024-11-13T05:06:15.444ZINFOSledAgent (VmmReservoirManager): Finished setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:279 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
2812024-11-13T05:06:15.480ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }
2822024-11-13T05:06:15.488ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:240
2832024-11-13T05:06:15.488ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:242
2842024-11-13T05:06:15.488ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:246
2852024-11-13T05:06:15.488ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:248
2862024-11-13T05:06:15.488ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2872024-11-13T05:06:15.488ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2882024-11-13T05:06:15.488ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
2892024-11-13T05:06:15.531ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2902024-11-13T05:06:15.538ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/zone file = sled-storage/src/dataset.rs:303
2912024-11-13T05:06:15.743ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_8511034a-a894-457f-9cbc-98e6461f9f5a file = sled-storage/src/dataset.rs:389
2922024-11-13T05:06:15.750ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:440
2932024-11-13T05:06:15.750ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:357
2942024-11-13T05:06:15.750ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:342
2952024-11-13T05:06:15.750ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2962024-11-13T05:06:15.750ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } path = /scratch/u2_1.vdev
2972024-11-13T05:06:15.759ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_1.vdev file = sled-hardware/src/disk.rs:367
2982024-11-13T05:06:15.759ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool)
2992024-11-13T05:06:15.864ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }
3002024-11-13T05:06:15.870ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:240
3012024-11-13T05:06:15.870ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:242
3022024-11-13T05:06:15.870ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:246
3032024-11-13T05:06:15.870ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:248
3042024-11-13T05:06:15.870ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
3052024-11-13T05:06:15.870ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
3062024-11-13T05:06:15.870ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3072024-11-13T05:06:15.907ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3082024-11-13T05:06:15.914ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_c867bbf7-921c-4db1-887a-4002bcc6b446/crypt/zone file = sled-storage/src/dataset.rs:303
3092024-11-13T05:06:16.032ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/4d997277-f640-48d7-af56-6df261d5afe1) file = sled-agent/src/probe_manager.rs:231 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
3102024-11-13T05:06:16.109ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_c867bbf7-921c-4db1-887a-4002bcc6b446 file = sled-storage/src/dataset.rs:389
3112024-11-13T05:06:16.116ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:440
3122024-11-13T05:06:16.116ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:357
3132024-11-13T05:06:16.116ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:342
3142024-11-13T05:06:16.116ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3152024-11-13T05:06:16.116ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } path = /scratch/u2_2.vdev
3162024-11-13T05:06:16.126ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_2.vdev file = sled-hardware/src/disk.rs:367
3172024-11-13T05:06:16.126ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool)
3182024-11-13T05:06:16.237ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }
3192024-11-13T05:06:16.244ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:240
3202024-11-13T05:06:16.244ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:242
3212024-11-13T05:06:16.244ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:246
3222024-11-13T05:06:16.244ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:248
3232024-11-13T05:06:16.244ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3242024-11-13T05:06:16.245ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3252024-11-13T05:06:16.245ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3262024-11-13T05:06:16.285ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3272024-11-13T05:06:16.293ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/zone file = sled-storage/src/dataset.rs:303
3282024-11-13T05:06:16.494ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_204f40c4-4e09-4726-a368-a3da2ac1b831 file = sled-storage/src/dataset.rs:389
3292024-11-13T05:06:16.502ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:440
3302024-11-13T05:06:16.502ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:357
3312024-11-13T05:06:16.502ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:342
3322024-11-13T05:06:16.502ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3332024-11-13T05:06:16.502ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } path = /scratch/u2_3.vdev
3342024-11-13T05:06:16.511ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_3.vdev file = sled-hardware/src/disk.rs:367
3352024-11-13T05:06:16.511ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool)
3362024-11-13T05:06:16.620ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }
3372024-11-13T05:06:16.626ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:240
3382024-11-13T05:06:16.626ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:242
3392024-11-13T05:06:16.626ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:246
3402024-11-13T05:06:16.626ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:248
3412024-11-13T05:06:16.626ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3422024-11-13T05:06:16.626ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3432024-11-13T05:06:16.626ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3442024-11-13T05:06:16.667ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3452024-11-13T05:06:16.673ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_8e821c87-2d05-4e93-b160-49a645a177c5/crypt/zone file = sled-storage/src/dataset.rs:303
3462024-11-13T05:06:16.865ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_8e821c87-2d05-4e93-b160-49a645a177c5 file = sled-storage/src/dataset.rs:389
3472024-11-13T05:06:16.872ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:440
3482024-11-13T05:06:16.872ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:357
3492024-11-13T05:06:16.872ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:342
3502024-11-13T05:06:16.872ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3512024-11-13T05:06:16.872ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } path = /scratch/u2_4.vdev
3522024-11-13T05:06:16.882ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_4.vdev file = sled-hardware/src/disk.rs:367
3532024-11-13T05:06:16.882ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool)
3542024-11-13T05:06:16.993ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }
3552024-11-13T05:06:17.002ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:240
3562024-11-13T05:06:17.002ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:242
3572024-11-13T05:06:17.002ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:246
3582024-11-13T05:06:17.002ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:248
3592024-11-13T05:06:17.002ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3602024-11-13T05:06:17.002ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3612024-11-13T05:06:17.002ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3622024-11-13T05:06:17.034ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/4d997277-f640-48d7-af56-6df261d5afe1) file = sled-agent/src/probe_manager.rs:231 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
3632024-11-13T05:06:17.044ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3642024-11-13T05:06:17.053ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_e770fce0-3dd5-4678-9037-68ea781dbb30/crypt/zone file = sled-storage/src/dataset.rs:303
3652024-11-13T05:06:17.260ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_e770fce0-3dd5-4678-9037-68ea781dbb30 file = sled-storage/src/dataset.rs:389
3662024-11-13T05:06:17.267ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:440
3672024-11-13T05:06:17.268ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:357
3682024-11-13T05:06:17.268ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:342
3692024-11-13T05:06:17.268ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3702024-11-13T05:06:17.268ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } path = /scratch/u2_5.vdev
3712024-11-13T05:06:17.277ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_5.vdev file = sled-hardware/src/disk.rs:367
3722024-11-13T05:06:17.278ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool)
3732024-11-13T05:06:17.395ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }
3742024-11-13T05:06:17.403ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:240
3752024-11-13T05:06:17.403ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:242
3762024-11-13T05:06:17.403ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:246
3772024-11-13T05:06:17.403ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:248
3782024-11-13T05:06:17.403ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3792024-11-13T05:06:17.403ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3802024-11-13T05:06:17.403ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3812024-11-13T05:06:17.446ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3822024-11-13T05:06:17.453ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_c2ce9d45-3d62-4234-a9db-c1d5c8884adc/crypt/zone file = sled-storage/src/dataset.rs:303
3832024-11-13T05:06:17.661ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_c2ce9d45-3d62-4234-a9db-c1d5c8884adc file = sled-storage/src/dataset.rs:389
3842024-11-13T05:06:17.670ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:440
3852024-11-13T05:06:17.670ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:357
3862024-11-13T05:06:17.670ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:342
3872024-11-13T05:06:17.670ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3882024-11-13T05:06:17.670ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } path = /scratch/u2_6.vdev
3892024-11-13T05:06:17.679ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_6.vdev file = sled-hardware/src/disk.rs:367
3902024-11-13T05:06:17.679ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool)
3912024-11-13T05:06:17.802ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }
3922024-11-13T05:06:17.809ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:240
3932024-11-13T05:06:17.809ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:242
3942024-11-13T05:06:17.809ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:246
3952024-11-13T05:06:17.809ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:248
3962024-11-13T05:06:17.809ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3972024-11-13T05:06:17.810ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3982024-11-13T05:06:17.810ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3992024-11-13T05:06:17.853ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4002024-11-13T05:06:17.861ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_c9fdd7dd-67c5-4597-a857-03bc9769dc87/crypt/zone file = sled-storage/src/dataset.rs:303
4012024-11-13T05:06:18.037ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/4d997277-f640-48d7-af56-6df261d5afe1) file = sled-agent/src/probe_manager.rs:231 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
4022024-11-13T05:06:18.064ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_c9fdd7dd-67c5-4597-a857-03bc9769dc87 file = sled-storage/src/dataset.rs:389
4032024-11-13T05:06:18.072ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:440
4042024-11-13T05:06:18.072ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:357
4052024-11-13T05:06:18.072ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:342
4062024-11-13T05:06:18.072ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4072024-11-13T05:06:18.072ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } path = /scratch/u2_7.vdev
4082024-11-13T05:06:18.081ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_7.vdev file = sled-hardware/src/disk.rs:367
4092024-11-13T05:06:18.081ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 27d5a04d-ccb1-4219-851b-abc94499936f (zpool)
4102024-11-13T05:06:18.193ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External }
4112024-11-13T05:06:18.200ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:240
4122024-11-13T05:06:18.200ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:242
4132024-11-13T05:06:18.200ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:246
4142024-11-13T05:06:18.200ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:248
4152024-11-13T05:06:18.200ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4162024-11-13T05:06:18.200ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4172024-11-13T05:06:18.200ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
4182024-11-13T05:06:18.242ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4192024-11-13T05:06:18.249ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_27d5a04d-ccb1-4219-851b-abc94499936f/crypt/zone file = sled-storage/src/dataset.rs:303
4202024-11-13T05:06:18.445ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_27d5a04d-ccb1-4219-851b-abc94499936f file = sled-storage/src/dataset.rs:389
4212024-11-13T05:06:18.452ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:440
4222024-11-13T05:06:18.452ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:357
4232024-11-13T05:06:18.452ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:342
4242024-11-13T05:06:18.452ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4252024-11-13T05:06:18.452ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } path = /scratch/u2_8.vdev
4262024-11-13T05:06:18.463ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_8.vdev file = sled-hardware/src/disk.rs:367
4272024-11-13T05:06:18.463ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = c7851ba0-6692-4564-8a56-a39ae4767989 (zpool)
4282024-11-13T05:06:18.584ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }
4292024-11-13T05:06:18.592ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:240
4302024-11-13T05:06:18.592ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:242
4312024-11-13T05:06:18.592ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:246
4322024-11-13T05:06:18.592ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:248
4332024-11-13T05:06:18.592ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4342024-11-13T05:06:18.592ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4352024-11-13T05:06:18.592ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
4362024-11-13T05:06:18.633ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4372024-11-13T05:06:18.641ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_c7851ba0-6692-4564-8a56-a39ae4767989/crypt/zone file = sled-storage/src/dataset.rs:303
4382024-11-13T05:06:18.849ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_c7851ba0-6692-4564-8a56-a39ae4767989 file = sled-storage/src/dataset.rs:389
4392024-11-13T05:06:18.857ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:440
4402024-11-13T05:06:18.857ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:357
4412024-11-13T05:06:18.857ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4422024-11-13T05:06:18.857ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 3c73febf-14eb-49a3-9af1-023b1a81de43 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 147a026f-d4f7-4267-9015-fa2814ab1bfc (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
4432024-11-13T05:06:18.857ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
4442024-11-13T05:06:18.857ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
4452024-11-13T05:06:18.858ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4462024-11-13T05:06:18.858ZINFOSledAgent: physical disks ensure: Updated storage file = sled-agent/src/sled_agent.rs:888 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
4472024-11-13T05:06:18.858ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
4482024-11-13T05:06:18.858ZINFOSledAgent: physical disks ensure: Propagating new generation of disks file = sled-agent/src/sled_agent.rs:914 generation = Generation(1) sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
4492024-11-13T05:06:18.858ZINFOSledAgent: physical disks ensure: Updated storage monitor file = sled-agent/src/sled_agent.rs:919 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
4502024-11-13T05:06:18.858ZINFOSledAgent: physical disks ensure: Updated zone bundler file = sled-agent/src/sled_agent.rs:924 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
4512024-11-13T05:06:18.858ZINFOSledAgent (ProbeManager): use_only_these_disks: Processing new request file = sled-agent/src/probe_manager.rs:122 gen = Generation(1) sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
4522024-11-13T05:06:18.858ZINFOSledAgent: physical disks ensure: Updated probes file = sled-agent/src/sled_agent.rs:930 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
4532024-11-13T05:06:18.858ZINFOSledAgent (InstanceManager): use_only_these_disks: Processing new request file = sled-agent/src/instance_manager.rs:799 gen = Generation(1)
4542024-11-13T05:06:18.858ZINFOSledAgent: physical disks ensure: Updated instances file = sled-agent/src/sled_agent.rs:935 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
4552024-11-13T05:06:18.858ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 4423519 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:36484 req_id = e0a052e3-8dfc-4a28-923a-277bb2bb154b response_code = 200 uri = /omicron-physical-disks
4562024-11-13T05:06:18.890ZINFOSledAgent (RSS): attempting to set up sled's datasets: DatasetsConfig { generation: Generation(1), datasets: {03e68a31-3deb-43a0-a9cb-6c023a641ab6 (dataset): DatasetConfig { id: 03e68a31-3deb-43a0-a9cb-6c023a641ab6 (dataset), name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0753ab4f-4a16-4c79-a9a8-42e9c1280152 (dataset): DatasetConfig { id: 0753ab4f-4a16-4c79-a9a8-42e9c1280152 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 11eaa260-95ea-43b9-ab29-06cf48f037b6 (dataset): DatasetConfig { id: 11eaa260-95ea-43b9-ab29-06cf48f037b6 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 14b81dea-50ab-4cb3-8647-00f6c0e999ab (dataset): DatasetConfig { id: 14b81dea-50ab-4cb3-8647-00f6c0e999ab (dataset), name: DatasetName { pool_name: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, kind: TransientZone { name: "oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 230b7b96-302e-4a30-91e0-6c15695c4a73 (dataset): DatasetConfig { id: 230b7b96-302e-4a30-91e0-6c15695c4a73 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 234ef012-01ac-4ce6-8232-10ec57e37670 (dataset): DatasetConfig { id: 234ef012-01ac-4ce6-8232-10ec57e37670 (dataset), name: DatasetName { pool_name: ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2f374f6b-29a5-4d0b-83a2-192b742153f7 (dataset): DatasetConfig { id: 2f374f6b-29a5-4d0b-83a2-192b742153f7 (dataset), name: DatasetName { pool_name: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 3ccbfed9-6ae1-4db3-ba96-e938b208aec8 (dataset): DatasetConfig { id: 3ccbfed9-6ae1-4db3-ba96-e938b208aec8 (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3f56300e-7f2c-4108-9c08-04d6c6cd67fc (dataset): DatasetConfig { id: 3f56300e-7f2c-4108-9c08-04d6c6cd67fc (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 443e3305-81a5-4561-8319-d53c96d33eb1 (dataset): DatasetConfig { id: 443e3305-81a5-4561-8319-d53c96d33eb1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 515bbfab-468c-473f-ac62-6acbd856e520 (dataset): DatasetConfig { id: 515bbfab-468c-473f-ac62-6acbd856e520 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: TransientZone { name: "oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 51d743fa-4ea6-4c9a-90eb-67622976c390 (dataset): DatasetConfig { id: 51d743fa-4ea6-4c9a-90eb-67622976c390 (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5c340108-b866-4738-89d2-3a845be31c77 (dataset): DatasetConfig { id: 5c340108-b866-4738-89d2-3a845be31c77 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 5e6a538d-8d1b-4383-b47e-dd2cb7dc739e (dataset): DatasetConfig { id: 5e6a538d-8d1b-4383-b47e-dd2cb7dc739e (dataset), name: DatasetName { pool_name: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 62a156b6-8a5e-49d5-9655-0e0d2f2e9372 (dataset): DatasetConfig { id: 62a156b6-8a5e-49d5-9655-0e0d2f2e9372 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6a1627cf-9186-4ead-8e7c-802da1609439 (dataset): DatasetConfig { id: 6a1627cf-9186-4ead-8e7c-802da1609439 (dataset), name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6e024825-dec5-42a6-b44e-76cd960b28ce (dataset): DatasetConfig { id: 6e024825-dec5-42a6-b44e-76cd960b28ce (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 70181566-2201-448a-be3f-1614dfb1cbf7 (dataset): DatasetConfig { id: 70181566-2201-448a-be3f-1614dfb1cbf7 (dataset), name: DatasetName { pool_name: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 71162161-5d48-43cd-b5e4-9d2140123174 (dataset): DatasetConfig { id: 71162161-5d48-43cd-b5e4-9d2140123174 (dataset), name: DatasetName { pool_name: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 7351c5da-ed48-46bc-bb13-a39c7b46376e (dataset): DatasetConfig { id: 7351c5da-ed48-46bc-bb13-a39c7b46376e (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 74101824-0ffa-460f-9202-a4f0fd82d2c3 (dataset): DatasetConfig { id: 74101824-0ffa-460f-9202-a4f0fd82d2c3 (dataset), name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 825ebf44-d2bc-4578-b8fe-eb7c784b0e2a (dataset): DatasetConfig { id: 825ebf44-d2bc-4578-b8fe-eb7c784b0e2a (dataset), name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8574eaf6-08e5-4b2f-8c8e-7fd23f99e314 (dataset): DatasetConfig { id: 8574eaf6-08e5-4b2f-8c8e-7fd23f99e314 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8aeeb762-e6ea-4053-8ea4-d48ce0cf8b5b (dataset): DatasetConfig { id: 8aeeb762-e6ea-4053-8ea4-d48ce0cf8b5b (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8c3a4dc3-9ab9-442f-bb8a-1da726f4af15 (dataset): DatasetConfig { id: 8c3a4dc3-9ab9-442f-bb8a-1da726f4af15 (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 8ca696df-56a5-4083-bb58-ee14f96cc6bb (dataset): DatasetConfig { id: 8ca696df-56a5-4083-bb58-ee14f96cc6bb (dataset), name: DatasetName { pool_name: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 90eba306-3ab1-4ffc-afe9-e4a2f03adda4 (dataset): DatasetConfig { id: 90eba306-3ab1-4ffc-afe9-e4a2f03adda4 (dataset), name: DatasetName { pool_name: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 98384db9-c5e5-4688-96b9-64ce4d43e523 (dataset): DatasetConfig { id: 98384db9-c5e5-4688-96b9-64ce4d43e523 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 9f56cfa6-bd46-4569-bac1-19c7f7b7dfe1 (dataset): DatasetConfig { id: 9f56cfa6-bd46-4569-bac1-19c7f7b7dfe1 (dataset), name: DatasetName { pool_name: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a2d34d68-9258-4b5f-8c4c-399abf3c4a18 (dataset): DatasetConfig { id: a2d34d68-9258-4b5f-8c4c-399abf3c4a18 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: Clickhouse }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a7fc787f-184c-459b-8a58-5d07473a8c43 (dataset): DatasetConfig { id: a7fc787f-184c-459b-8a58-5d07473a8c43 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b014b1cf-1e08-44b7-9a68-12d5e5a59d1f (dataset): DatasetConfig { id: b014b1cf-1e08-44b7-9a68-12d5e5a59d1f (dataset), name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b37f0150-6ab5-4ea3-ae36-e90d3f238ff7 (dataset): DatasetConfig { id: b37f0150-6ab5-4ea3-ae36-e90d3f238ff7 (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b3a56fc2-493d-4ca8-949f-bdb2562dfde3 (dataset): DatasetConfig { id: b3a56fc2-493d-4ca8-949f-bdb2562dfde3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b76fa428-a15d-4be1-aea6-9e13fe9a9746 (dataset): DatasetConfig { id: b76fa428-a15d-4be1-aea6-9e13fe9a9746 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, bba280a9-b334-4be7-8d3f-61d4524568d4 (dataset): DatasetConfig { id: bba280a9-b334-4be7-8d3f-61d4524568d4 (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c07e3c0f-e6a5-408a-9d1f-4a1943cda734 (dataset): DatasetConfig { id: c07e3c0f-e6a5-408a-9d1f-4a1943cda734 (dataset), name: DatasetName { pool_name: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c3706e54-2742-4b1f-9980-2dc0666dbfc9 (dataset): DatasetConfig { id: c3706e54-2742-4b1f-9980-2dc0666dbfc9 (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c379281e-5645-4622-a38e-915fd3d871da (dataset): DatasetConfig { id: c379281e-5645-4622-a38e-915fd3d871da (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c6964afa-2b78-4b21-912b-5820db825c75 (dataset): DatasetConfig { id: c6964afa-2b78-4b21-912b-5820db825c75 (dataset), name: DatasetName { pool_name: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ce6cb9df-285e-4f2f-b627-6ae28082b269 (dataset): DatasetConfig { id: ce6cb9df-285e-4f2f-b627-6ae28082b269 (dataset), name: DatasetName { pool_name: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, cfc6b20c-5761-4ceb-a75c-a1442e83b839 (dataset): DatasetConfig { id: cfc6b20c-5761-4ceb-a75c-a1442e83b839 (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d0344ecd-7757-43bf-a4d5-30679c09fdc9 (dataset): DatasetConfig { id: d0344ecd-7757-43bf-a4d5-30679c09fdc9 (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d16d45f3-11e0-4a3c-88e5-00042c1143ef (dataset): DatasetConfig { id: d16d45f3-11e0-4a3c-88e5-00042c1143ef (dataset), name: DatasetName { pool_name: ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, d2421d60-f4d3-4b88-af8d-a58779d99a4e (dataset): DatasetConfig { id: d2421d60-f4d3-4b88-af8d-a58779d99a4e (dataset), name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d569f957-4bdf-4e66-b1f6-ba5083ad030b (dataset): DatasetConfig { id: d569f957-4bdf-4e66-b1f6-ba5083ad030b (dataset), name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d7a65f58-1bd5-4844-b33e-8614fcb6caae (dataset): DatasetConfig { id: d7a65f58-1bd5-4844-b33e-8614fcb6caae (dataset), name: DatasetName { pool_name: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, da38a58f-2733-4e81-b976-a11f842f2214 (dataset): DatasetConfig { id: da38a58f-2733-4e81-b976-a11f842f2214 (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, daefeba1-c981-4f38-8c27-e1a21daa0273 (dataset): DatasetConfig { id: daefeba1-c981-4f38-8c27-e1a21daa0273 (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, db435a3a-225c-492b-af38-dda6ea7ec7bf (dataset): DatasetConfig { id: db435a3a-225c-492b-af38-dda6ea7ec7bf (dataset), name: DatasetName { pool_name: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, db67ec02-40e9-4856-98ef-4663ad1dbcb0 (dataset): DatasetConfig { id: db67ec02-40e9-4856-98ef-4663ad1dbcb0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e2c1701c-b374-40ed-bdf5-3208eea13699 (dataset): DatasetConfig { id: e2c1701c-b374-40ed-bdf5-3208eea13699 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e57e5716-f03d-4bd5-9719-d6ce385b5b2e (dataset): DatasetConfig { id: e57e5716-f03d-4bd5-9719-d6ce385b5b2e (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ebde603b-255c-4083-8f0c-de88f7423e81 (dataset): DatasetConfig { id: ebde603b-255c-4083-8f0c-de88f7423e81 (dataset), name: DatasetName { pool_name: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ed7b2458-b4be-4565-b867-0f5f0301bbbc (dataset): DatasetConfig { id: ed7b2458-b4be-4565-b867-0f5f0301bbbc (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ef229db4-97ea-47ef-a1db-3eac8a835349 (dataset): DatasetConfig { id: ef229db4-97ea-47ef-a1db-3eac8a835349 (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f0121244-baad-483b-8c98-5e2fe21b651e (dataset): DatasetConfig { id: f0121244-baad-483b-8c98-5e2fe21b651e (dataset), name: DatasetName { pool_name: ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f1c9654c-1312-465f-8332-d7110f72b3bd (dataset): DatasetConfig { id: f1c9654c-1312-465f-8332-d7110f72b3bd (dataset), name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f4144f4e-9af9-4afd-8d8d-51a2fb32cae7 (dataset): DatasetConfig { id: f4144f4e-9af9-4afd-8d8d-51a2fb32cae7 (dataset), name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f450a5b9-6347-4f1c-ae67-e6af28c35ee5 (dataset): DatasetConfig { id: f450a5b9-6347-4f1c-ae67-e6af28c35ee5 (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f4e4c25a-acba-48cc-afba-7a24644e61bf (dataset): DatasetConfig { id: f4e4c25a-acba-48cc-afba-7a24644e61bf (dataset), name: DatasetName { pool_name: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f83b9ce1-a722-4a2f-8aca-33b11fc76722 (dataset): DatasetConfig { id: f83b9ce1-a722-4a2f-8aca-33b11fc76722 (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: TransientZone { name: "oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f9ec5400-dfb6-4a67-9ef0-10a053a1c88f (dataset): DatasetConfig { id: f9ec5400-dfb6-4a67-9ef0-10a053a1c88f (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, fadd2fcc-b60c-47f7-9273-93c13b51eb55 (dataset): DatasetConfig { id: fadd2fcc-b60c-47f7-9273-93c13b51eb55 (dataset), name: DatasetName { pool_name: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, fce79aea-f59c-4723-baeb-0a0351c84420 (dataset): DatasetConfig { id: fce79aea-f59c-4723-baeb-0a0351c84420 (dataset), name: DatasetName { pool_name: ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, fe3bec39-9ecb-4e38-900b-0134fd5a25dc (dataset): DatasetConfig { id: fe3bec39-9ecb-4e38-900b-0134fd5a25dc (dataset), name: DatasetName { pool_name: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }} } file = sled-agent/src/rack_setup/service.rs:457 sled_address = [fd00:1122:3344:101::1]:12345
4572024-11-13T05:06:18.890ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57489
4582024-11-13T05:06:18.890ZINFOSledAgent: datasets ensure file = sled-agent/src/sled_agent.rs:850 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
4592024-11-13T05:06:18.891ZINFOSledAgent (StorageManager): Received DatasetsEnsure { config: DatasetsConfig { generation: Generation(1), datasets: {03e68a31-3deb-43a0-a9cb-6c023a641ab6 (dataset): DatasetConfig { id: 03e68a31-3deb-43a0-a9cb-6c023a641ab6 (dataset), name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0753ab4f-4a16-4c79-a9a8-42e9c1280152 (dataset): DatasetConfig { id: 0753ab4f-4a16-4c79-a9a8-42e9c1280152 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 11eaa260-95ea-43b9-ab29-06cf48f037b6 (dataset): DatasetConfig { id: 11eaa260-95ea-43b9-ab29-06cf48f037b6 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 14b81dea-50ab-4cb3-8647-00f6c0e999ab (dataset): DatasetConfig { id: 14b81dea-50ab-4cb3-8647-00f6c0e999ab (dataset), name: DatasetName { pool_name: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, kind: TransientZone { name: "oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 230b7b96-302e-4a30-91e0-6c15695c4a73 (dataset): DatasetConfig { id: 230b7b96-302e-4a30-91e0-6c15695c4a73 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 234ef012-01ac-4ce6-8232-10ec57e37670 (dataset): DatasetConfig { id: 234ef012-01ac-4ce6-8232-10ec57e37670 (dataset), name: DatasetName { pool_name: ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2f374f6b-29a5-4d0b-83a2-192b742153f7 (dataset): DatasetConfig { id: 2f374f6b-29a5-4d0b-83a2-192b742153f7 (dataset), name: DatasetName { pool_name: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 3ccbfed9-6ae1-4db3-ba96-e938b208aec8 (dataset): DatasetConfig { id: 3ccbfed9-6ae1-4db3-ba96-e938b208aec8 (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3f56300e-7f2c-4108-9c08-04d6c6cd67fc (dataset): DatasetConfig { id: 3f56300e-7f2c-4108-9c08-04d6c6cd67fc (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 443e3305-81a5-4561-8319-d53c96d33eb1 (dataset): DatasetConfig { id: 443e3305-81a5-4561-8319-d53c96d33eb1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 515bbfab-468c-473f-ac62-6acbd856e520 (dataset): DatasetConfig { id: 515bbfab-468c-473f-ac62-6acbd856e520 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: TransientZone { name: "oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 51d743fa-4ea6-4c9a-90eb-67622976c390 (dataset): DatasetConfig { id: 51d743fa-4ea6-4c9a-90eb-67622976c390 (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5c340108-b866-4738-89d2-3a845be31c77 (dataset): DatasetConfig { id: 5c340108-b866-4738-89d2-3a845be31c77 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 5e6a538d-8d1b-4383-b47e-dd2cb7dc739e (dataset): DatasetConfig { id: 5e6a538d-8d1b-4383-b47e-dd2cb7dc739e (dataset), name: DatasetName { pool_name: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 62a156b6-8a5e-49d5-9655-0e0d2f2e9372 (dataset): DatasetConfig { id: 62a156b6-8a5e-49d5-9655-0e0d2f2e9372 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6a1627cf-9186-4ead-8e7c-802da1609439 (dataset): DatasetConfig { id: 6a1627cf-9186-4ead-8e7c-802da1609439 (dataset), name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6e024825-dec5-42a6-b44e-76cd960b28ce (dataset): DatasetConfig { id: 6e024825-dec5-42a6-b44e-76cd960b28ce (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 70181566-2201-448a-be3f-1614dfb1cbf7 (dataset): DatasetConfig { id: 70181566-2201-448a-be3f-1614dfb1cbf7 (dataset), name: DatasetName { pool_name: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 71162161-5d48-43cd-b5e4-9d2140123174 (dataset): DatasetConfig { id: 71162161-5d48-43cd-b5e4-9d2140123174 (dataset), name: DatasetName { pool_name: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 7351c5da-ed48-46bc-bb13-a39c7b46376e (dataset): DatasetConfig { id: 7351c5da-ed48-46bc-bb13-a39c7b46376e (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 74101824-0ffa-460f-9202-a4f0fd82d2c3 (dataset): DatasetConfig { id: 74101824-0ffa-460f-9202-a4f0fd82d2c3 (dataset), name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 825ebf44-d2bc-4578-b8fe-eb7c784b0e2a (dataset): DatasetConfig { id: 825ebf44-d2bc-4578-b8fe-eb7c784b0e2a (dataset), name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8574eaf6-08e5-4b2f-8c8e-7fd23f99e314 (dataset): DatasetConfig { id: 8574eaf6-08e5-4b2f-8c8e-7fd23f99e314 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8aeeb762-e6ea-4053-8ea4-d48ce0cf8b5b (dataset): DatasetConfig { id: 8aeeb762-e6ea-4053-8ea4-d48ce0cf8b5b (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8c3a4dc3-9ab9-442f-bb8a-1da726f4af15 (dataset): DatasetConfig { id: 8c3a4dc3-9ab9-442f-bb8a-1da726f4af15 (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 8ca696df-56a5-4083-bb58-ee14f96cc6bb (dataset): DatasetConfig { id: 8ca696df-56a5-4083-bb58-ee14f96cc6bb (dataset), name: DatasetName { pool_name: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 90eba306-3ab1-4ffc-afe9-e4a2f03adda4 (dataset): DatasetConfig { id: 90eba306-3ab1-4ffc-afe9-e4a2f03adda4 (dataset), name: DatasetName { pool_name: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 98384db9-c5e5-4688-96b9-64ce4d43e523 (dataset): DatasetConfig { id: 98384db9-c5e5-4688-96b9-64ce4d43e523 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 9f56cfa6-bd46-4569-bac1-19c7f7b7dfe1 (dataset): DatasetConfig { id: 9f56cfa6-bd46-4569-bac1-19c7f7b7dfe1 (dataset), name: DatasetName { pool_name: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a2d34d68-9258-4b5f-8c4c-399abf3c4a18 (dataset): DatasetConfig { id: a2d34d68-9258-4b5f-8c4c-399abf3c4a18 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: Clickhouse }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a7fc787f-184c-459b-8a58-5d07473a8c43 (dataset): DatasetConfig { id: a7fc787f-184c-459b-8a58-5d07473a8c43 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b014b1cf-1e08-44b7-9a68-12d5e5a59d1f (dataset): DatasetConfig { id: b014b1cf-1e08-44b7-9a68-12d5e5a59d1f (dataset), name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b37f0150-6ab5-4ea3-ae36-e90d3f238ff7 (dataset): DatasetConfig { id: b37f0150-6ab5-4ea3-ae36-e90d3f238ff7 (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b3a56fc2-493d-4ca8-949f-bdb2562dfde3 (dataset): DatasetConfig { id: b3a56fc2-493d-4ca8-949f-bdb2562dfde3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b76fa428-a15d-4be1-aea6-9e13fe9a9746 (dataset): DatasetConfig { id: b76fa428-a15d-4be1-aea6-9e13fe9a9746 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, bba280a9-b334-4be7-8d3f-61d4524568d4 (dataset): DatasetConfig { id: bba280a9-b334-4be7-8d3f-61d4524568d4 (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c07e3c0f-e6a5-408a-9d1f-4a1943cda734 (dataset): DatasetConfig { id: c07e3c0f-e6a5-408a-9d1f-4a1943cda734 (dataset), name: DatasetName { pool_name: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c3706e54-2742-4b1f-9980-2dc0666dbfc9 (dataset): DatasetConfig { id: c3706e54-2742-4b1f-9980-2dc0666dbfc9 (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c379281e-5645-4622-a38e-915fd3d871da (dataset): DatasetConfig { id: c379281e-5645-4622-a38e-915fd3d871da (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c6964afa-2b78-4b21-912b-5820db825c75 (dataset): DatasetConfig { id: c6964afa-2b78-4b21-912b-5820db825c75 (dataset), name: DatasetName { pool_name: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ce6cb9df-285e-4f2f-b627-6ae28082b269 (dataset): DatasetConfig { id: ce6cb9df-285e-4f2f-b627-6ae28082b269 (dataset), name: DatasetName { pool_name: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, cfc6b20c-5761-4ceb-a75c-a1442e83b839 (dataset): DatasetConfig { id: cfc6b20c-5761-4ceb-a75c-a1442e83b839 (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d0344ecd-7757-43bf-a4d5-30679c09fdc9 (dataset): DatasetConfig { id: d0344ecd-7757-43bf-a4d5-30679c09fdc9 (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d16d45f3-11e0-4a3c-88e5-00042c1143ef (dataset): DatasetConfig { id: d16d45f3-11e0-4a3c-88e5-00042c1143ef (dataset), name: DatasetName { pool_name: ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, d2421d60-f4d3-4b88-af8d-a58779d99a4e (dataset): DatasetConfig { id: d2421d60-f4d3-4b88-af8d-a58779d99a4e (dataset), name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d569f957-4bdf-4e66-b1f6-ba5083ad030b (dataset): DatasetConfig { id: d569f957-4bdf-4e66-b1f6-ba5083ad030b (dataset), name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d7a65f58-1bd5-4844-b33e-8614fcb6caae (dataset): DatasetConfig { id: d7a65f58-1bd5-4844-b33e-8614fcb6caae (dataset), name: DatasetName { pool_name: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, da38a58f-2733-4e81-b976-a11f842f2214 (dataset): DatasetConfig { id: da38a58f-2733-4e81-b976-a11f842f2214 (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, daefeba1-c981-4f38-8c27-e1a21daa0273 (dataset): DatasetConfig { id: daefeba1-c981-4f38-8c27-e1a21daa0273 (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, db435a3a-225c-492b-af38-dda6ea7ec7bf (dataset): DatasetConfig { id: db435a3a-225c-492b-af38-dda6ea7ec7bf (dataset), name: DatasetName { pool_name: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, db67ec02-40e9-4856-98ef-4663ad1dbcb0 (dataset): DatasetConfig { id: db67ec02-40e9-4856-98ef-4663ad1dbcb0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e2c1701c-b374-40ed-bdf5-3208eea13699 (dataset): DatasetConfig { id: e2c1701c-b374-40ed-bdf5-3208eea13699 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e57e5716-f03d-4bd5-9719-d6ce385b5b2e (dataset): DatasetConfig { id: e57e5716-f03d-4bd5-9719-d6ce385b5b2e (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ebde603b-255c-4083-8f0c-de88f7423e81 (dataset): DatasetConfig { id: ebde603b-255c-4083-8f0c-de88f7423e81 (dataset), name: DatasetName { pool_name: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ed7b2458-b4be-4565-b867-0f5f0301bbbc (dataset): DatasetConfig { id: ed7b2458-b4be-4565-b867-0f5f0301bbbc (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ef229db4-97ea-47ef-a1db-3eac8a835349 (dataset): DatasetConfig { id: ef229db4-97ea-47ef-a1db-3eac8a835349 (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f0121244-baad-483b-8c98-5e2fe21b651e (dataset): DatasetConfig { id: f0121244-baad-483b-8c98-5e2fe21b651e (dataset), name: DatasetName { pool_name: ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f1c9654c-1312-465f-8332-d7110f72b3bd (dataset): DatasetConfig { id: f1c9654c-1312-465f-8332-d7110f72b3bd (dataset), name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f4144f4e-9af9-4afd-8d8d-51a2fb32cae7 (dataset): DatasetConfig { id: f4144f4e-9af9-4afd-8d8d-51a2fb32cae7 (dataset), name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f450a5b9-6347-4f1c-ae67-e6af28c35ee5 (dataset): DatasetConfig { id: f450a5b9-6347-4f1c-ae67-e6af28c35ee5 (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f4e4c25a-acba-48cc-afba-7a24644e61bf (dataset): DatasetConfig { id: f4e4c25a-acba-48cc-afba-7a24644e61bf (dataset), name: DatasetName { pool_name: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f83b9ce1-a722-4a2f-8aca-33b11fc76722 (dataset): DatasetConfig { id: f83b9ce1-a722-4a2f-8aca-33b11fc76722 (dataset), name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: TransientZone { name: "oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f9ec5400-dfb6-4a67-9ef0-10a053a1c88f (dataset): DatasetConfig { id: f9ec5400-dfb6-4a67-9ef0-10a053a1c88f (dataset), name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, fadd2fcc-b60c-47f7-9273-93c13b51eb55 (dataset): DatasetConfig { id: fadd2fcc-b60c-47f7-9273-93c13b51eb55 (dataset), name: DatasetName { pool_name: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, fce79aea-f59c-4723-baeb-0a0351c84420 (dataset): DatasetConfig { id: fce79aea-f59c-4723-baeb-0a0351c84420 (dataset), name: DatasetName { pool_name: ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, fe3bec39-9ecb-4e38-900b-0134fd5a25dc (dataset): DatasetConfig { id: fe3bec39-9ecb-4e38-900b-0134fd5a25dc (dataset), name: DatasetName { pool_name: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }} }, tx: ... } file = sled-storage/src/manager.rs:603
4602024-11-13T05:06:18.891ZINFOSledAgent (StorageManager): No ledger in /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/omicron-datasets.json file = common/src/ledger.rs:188 request = datasets_ensure
4612024-11-13T05:06:18.891ZINFOSledAgent (StorageManager): No ledger in /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/omicron-datasets.json file = common/src/ledger.rs:188 request = datasets_ensure
4622024-11-13T05:06:18.891ZINFOSledAgent (StorageManager): No previously-stored 'requested datasets', creating new ledger file = sled-storage/src/manager.rs:905 request = datasets_ensure
4632024-11-13T05:06:18.891ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4642024-11-13T05:06:18.891ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/e770fce0-3dd5-4678-9037-68ea781dbb30/crypt/zone"), full_name: "oxp_e770fce0-3dd5-4678-9037-68ea781dbb30/crypt/zone" } file = sled-storage/src/manager.rs:1331
4652024-11-13T05:06:18.971ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4662024-11-13T05:06:18.971ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_8e821c87-2d05-4e93-b160-49a645a177c5/crucible" } file = sled-storage/src/manager.rs:1331
4672024-11-13T05:06:19.040ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/4d997277-f640-48d7-af56-6df261d5afe1) file = sled-agent/src/probe_manager.rs:231 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
4682024-11-13T05:06:19.079ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4692024-11-13T05:06:19.079ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/internal_dns" } file = sled-storage/src/manager.rs:1331
4702024-11-13T05:06:19.188ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4712024-11-13T05:06:19.188ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c2ce9d45-3d62-4234-a9db-c1d5c8884adc/crypt/zone/oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583"), full_name: "oxp_c2ce9d45-3d62-4234-a9db-c1d5c8884adc/crypt/zone/oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583" } file = sled-storage/src/manager.rs:1331
4722024-11-13T05:06:19.306ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4732024-11-13T05:06:19.306ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/zone/oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44"), full_name: "oxp_8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/zone/oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44" } file = sled-storage/src/manager.rs:1331
4742024-11-13T05:06:19.422ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4752024-11-13T05:06:19.422ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/27d5a04d-ccb1-4219-851b-abc94499936f/crypt/zone"), full_name: "oxp_27d5a04d-ccb1-4219-851b-abc94499936f/crypt/zone" } file = sled-storage/src/manager.rs:1331
4762024-11-13T05:06:19.504ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4772024-11-13T05:06:19.504ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c2ce9d45-3d62-4234-a9db-c1d5c8884adc/crypt/debug"), full_name: "oxp_c2ce9d45-3d62-4234-a9db-c1d5c8884adc/crypt/debug" } file = sled-storage/src/manager.rs:1331
4782024-11-13T05:06:19.583ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4792024-11-13T05:06:19.583ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c867bbf7-921c-4db1-887a-4002bcc6b446/crypt/zone/oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6"), full_name: "oxp_c867bbf7-921c-4db1-887a-4002bcc6b446/crypt/zone/oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6" } file = sled-storage/src/manager.rs:1331
4802024-11-13T05:06:19.700ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4812024-11-13T05:06:19.700ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/zone"), full_name: "oxp_8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/zone" } file = sled-storage/src/manager.rs:1331
4822024-11-13T05:06:19.779ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4832024-11-13T05:06:19.779ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/zone/oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a"), full_name: "oxp_204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/zone/oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a" } file = sled-storage/src/manager.rs:1331
4842024-11-13T05:06:19.893ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4852024-11-13T05:06:19.893ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/zone/oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c"), full_name: "oxp_8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/zone/oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c" } file = sled-storage/src/manager.rs:1331
4862024-11-13T05:06:20.008ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4872024-11-13T05:06:20.008ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c867bbf7-921c-4db1-887a-4002bcc6b446/crypt/zone/oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca"), full_name: "oxp_c867bbf7-921c-4db1-887a-4002bcc6b446/crypt/zone/oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca" } file = sled-storage/src/manager.rs:1331
4882024-11-13T05:06:20.042ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/4d997277-f640-48d7-af56-6df261d5afe1) file = sled-agent/src/probe_manager.rs:231 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
4892024-11-13T05:06:20.124ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4902024-11-13T05:06:20.124ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/debug"), full_name: "oxp_8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/debug" } file = sled-storage/src/manager.rs:1331
4912024-11-13T05:06:20.203ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4922024-11-13T05:06:20.203ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c7851ba0-6692-4564-8a56-a39ae4767989/crypt/zone"), full_name: "oxp_c7851ba0-6692-4564-8a56-a39ae4767989/crypt/zone" } file = sled-storage/src/manager.rs:1331
4932024-11-13T05:06:20.282ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4942024-11-13T05:06:20.282ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/external_dns" } file = sled-storage/src/manager.rs:1331
4952024-11-13T05:06:20.390ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4962024-11-13T05:06:20.390ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/e770fce0-3dd5-4678-9037-68ea781dbb30/crypt/zone/oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5"), full_name: "oxp_e770fce0-3dd5-4678-9037-68ea781dbb30/crypt/zone/oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5" } file = sled-storage/src/manager.rs:1331
4972024-11-13T05:06:20.504ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4982024-11-13T05:06:20.504ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
4992024-11-13T05:06:20.611ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5002024-11-13T05:06:20.611ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c7851ba0-6692-4564-8a56-a39ae4767989/crypt/zone/oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f"), full_name: "oxp_c7851ba0-6692-4564-8a56-a39ae4767989/crypt/zone/oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f" } file = sled-storage/src/manager.rs:1331
5012024-11-13T05:06:20.726ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5022024-11-13T05:06:20.726ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c9fdd7dd-67c5-4597-a857-03bc9769dc87/crypt/debug"), full_name: "oxp_c9fdd7dd-67c5-4597-a857-03bc9769dc87/crypt/debug" } file = sled-storage/src/manager.rs:1331
5032024-11-13T05:06:20.803ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5042024-11-13T05:06:20.803ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/zone/oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474"), full_name: "oxp_8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/zone/oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474" } file = sled-storage/src/manager.rs:1331
5052024-11-13T05:06:20.915ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5062024-11-13T05:06:20.915ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/e770fce0-3dd5-4678-9037-68ea781dbb30/crypt/debug"), full_name: "oxp_e770fce0-3dd5-4678-9037-68ea781dbb30/crypt/debug" } file = sled-storage/src/manager.rs:1331
5072024-11-13T05:06:20.992ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5082024-11-13T05:06:20.992ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_8e821c87-2d05-4e93-b160-49a645a177c5/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
5092024-11-13T05:06:21.047ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/4d997277-f640-48d7-af56-6df261d5afe1) file = sled-agent/src/probe_manager.rs:231 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
5102024-11-13T05:06:21.101ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5112024-11-13T05:06:21.101ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8e821c87-2d05-4e93-b160-49a645a177c5/crypt/zone/oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525"), full_name: "oxp_8e821c87-2d05-4e93-b160-49a645a177c5/crypt/zone/oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525" } file = sled-storage/src/manager.rs:1331
5122024-11-13T05:06:21.218ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5132024-11-13T05:06:21.218ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/zone"), full_name: "oxp_204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/zone" } file = sled-storage/src/manager.rs:1331
5142024-11-13T05:06:21.298ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5152024-11-13T05:06:21.298ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/debug"), full_name: "oxp_204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/debug" } file = sled-storage/src/manager.rs:1331
5162024-11-13T05:06:21.378ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5172024-11-13T05:06:21.378ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_c9fdd7dd-67c5-4597-a857-03bc9769dc87/crucible" } file = sled-storage/src/manager.rs:1331
5182024-11-13T05:06:21.485ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5192024-11-13T05:06:21.485ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c9fdd7dd-67c5-4597-a857-03bc9769dc87/crypt/zone/oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237"), full_name: "oxp_c9fdd7dd-67c5-4597-a857-03bc9769dc87/crypt/zone/oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237" } file = sled-storage/src/manager.rs:1331
5202024-11-13T05:06:21.597ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5212024-11-13T05:06:21.597ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8e821c87-2d05-4e93-b160-49a645a177c5/crypt/debug"), full_name: "oxp_8e821c87-2d05-4e93-b160-49a645a177c5/crypt/debug" } file = sled-storage/src/manager.rs:1331
5222024-11-13T05:06:21.679ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5232024-11-13T05:06:21.679ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c2ce9d45-3d62-4234-a9db-c1d5c8884adc/crypt/zone"), full_name: "oxp_c2ce9d45-3d62-4234-a9db-c1d5c8884adc/crypt/zone" } file = sled-storage/src/manager.rs:1331
5242024-11-13T05:06:21.758ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5252024-11-13T05:06:21.758ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/clickhouse" } file = sled-storage/src/manager.rs:1331
5262024-11-13T05:06:21.867ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5272024-11-13T05:06:21.867ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8e821c87-2d05-4e93-b160-49a645a177c5/crypt/zone/oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641"), full_name: "oxp_8e821c87-2d05-4e93-b160-49a645a177c5/crypt/zone/oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641" } file = sled-storage/src/manager.rs:1331
5282024-11-13T05:06:21.983ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5292024-11-13T05:06:21.983ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/e770fce0-3dd5-4678-9037-68ea781dbb30/crypt/zone/oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca"), full_name: "oxp_e770fce0-3dd5-4678-9037-68ea781dbb30/crypt/zone/oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca" } file = sled-storage/src/manager.rs:1331
5302024-11-13T05:06:22.049ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/4d997277-f640-48d7-af56-6df261d5afe1) file = sled-agent/src/probe_manager.rs:231 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
5312024-11-13T05:06:22.100ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5322024-11-13T05:06:22.100ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_c867bbf7-921c-4db1-887a-4002bcc6b446/crypt/internal_dns" } file = sled-storage/src/manager.rs:1331
5332024-11-13T05:06:22.208ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5342024-11-13T05:06:22.208ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/zone/oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208"), full_name: "oxp_8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/zone/oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208" } file = sled-storage/src/manager.rs:1331
5352024-11-13T05:06:22.321ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5362024-11-13T05:06:22.321ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
5372024-11-13T05:06:22.430ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5382024-11-13T05:06:22.430ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/internal_dns" } file = sled-storage/src/manager.rs:1331
5392024-11-13T05:06:22.537ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5402024-11-13T05:06:22.538ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c9fdd7dd-67c5-4597-a857-03bc9769dc87/crypt/zone/oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb"), full_name: "oxp_c9fdd7dd-67c5-4597-a857-03bc9769dc87/crypt/zone/oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb" } file = sled-storage/src/manager.rs:1331
5412024-11-13T05:06:22.655ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5422024-11-13T05:06:22.655ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c867bbf7-921c-4db1-887a-4002bcc6b446/crypt/zone/oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317"), full_name: "oxp_c867bbf7-921c-4db1-887a-4002bcc6b446/crypt/zone/oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317" } file = sled-storage/src/manager.rs:1331
5432024-11-13T05:06:22.801ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5442024-11-13T05:06:22.801ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_204f40c4-4e09-4726-a368-a3da2ac1b831/crucible" } file = sled-storage/src/manager.rs:1331
5452024-11-13T05:06:22.910ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5462024-11-13T05:06:22.910ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c7851ba0-6692-4564-8a56-a39ae4767989/crypt/zone/oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5"), full_name: "oxp_c7851ba0-6692-4564-8a56-a39ae4767989/crypt/zone/oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5" } file = sled-storage/src/manager.rs:1331
5472024-11-13T05:06:23.029ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5482024-11-13T05:06:23.029ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c2ce9d45-3d62-4234-a9db-c1d5c8884adc/crypt/zone/oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be"), full_name: "oxp_c2ce9d45-3d62-4234-a9db-c1d5c8884adc/crypt/zone/oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be" } file = sled-storage/src/manager.rs:1331
5492024-11-13T05:06:23.051ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/4d997277-f640-48d7-af56-6df261d5afe1) file = sled-agent/src/probe_manager.rs:231 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
5502024-11-13T05:06:23.149ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5512024-11-13T05:06:23.149ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_c867bbf7-921c-4db1-887a-4002bcc6b446/crypt/external_dns" } file = sled-storage/src/manager.rs:1331
5522024-11-13T05:06:23.166ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 21.073083849s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:391 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
5532024-11-13T05:06:23.257ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5542024-11-13T05:06:23.258ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/zone/oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be"), full_name: "oxp_204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/zone/oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be" } file = sled-storage/src/manager.rs:1331
5552024-11-13T05:06:23.377ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5562024-11-13T05:06:23.378ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/27d5a04d-ccb1-4219-851b-abc94499936f/crypt/debug"), full_name: "oxp_27d5a04d-ccb1-4219-851b-abc94499936f/crypt/debug" } file = sled-storage/src/manager.rs:1331
5572024-11-13T05:06:23.458ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5582024-11-13T05:06:23.458ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_e770fce0-3dd5-4678-9037-68ea781dbb30/crucible" } file = sled-storage/src/manager.rs:1331
5592024-11-13T05:06:23.567ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5602024-11-13T05:06:23.567ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8e821c87-2d05-4e93-b160-49a645a177c5/crypt/zone"), full_name: "oxp_8e821c87-2d05-4e93-b160-49a645a177c5/crypt/zone" } file = sled-storage/src/manager.rs:1331
5612024-11-13T05:06:23.647ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5622024-11-13T05:06:23.647ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_c7851ba0-6692-4564-8a56-a39ae4767989/crucible" } file = sled-storage/src/manager.rs:1331
5632024-11-13T05:06:23.758ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5642024-11-13T05:06:23.758ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_c867bbf7-921c-4db1-887a-4002bcc6b446/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
5652024-11-13T05:06:23.868ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5662024-11-13T05:06:23.868ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/zone/oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148"), full_name: "oxp_204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/zone/oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148" } file = sled-storage/src/manager.rs:1331
5672024-11-13T05:06:23.919ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 5578476604298381371 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
5682024-11-13T05:06:23.919ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8090005703617564393 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
5692024-11-13T05:06:23.983ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5702024-11-13T05:06:23.983ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c7851ba0-6692-4564-8a56-a39ae4767989/crypt/debug"), full_name: "oxp_c7851ba0-6692-4564-8a56-a39ae4767989/crypt/debug" } file = sled-storage/src/manager.rs:1331
5712024-11-13T05:06:24.057ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/4d997277-f640-48d7-af56-6df261d5afe1) file = sled-agent/src/probe_manager.rs:231 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
5722024-11-13T05:06:24.068ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5732024-11-13T05:06:24.068ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/zone/oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66"), full_name: "oxp_8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/zone/oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66" } file = sled-storage/src/manager.rs:1331
5742024-11-13T05:06:24.161ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists - OK file = sled-agent/src/services.rs:4292
5752024-11-13T05:06:24.161ZINFOSledAgent (ServiceManager): Ensuring there is a default route file = sled-agent/src/services.rs:4302 gateway = fd00:1122:3344:101::1
5762024-11-13T05:06:24.167ZINFOSledAgent (ServiceManager): Default route already exists file = sled-agent/src/services.rs:4316 gateway = fd00:1122:3344:101::1
5772024-11-13T05:06:24.167ZINFOSledAgent (ServiceManager): configuring dendrite service file = sled-agent/src/services.rs:4381
5782024-11-13T05:06:24.193ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5792024-11-13T05:06:24.193ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_8511034a-a894-457f-9cbc-98e6461f9f5a/crucible" } file = sled-storage/src/manager.rs:1331
5802024-11-13T05:06:24.310ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5812024-11-13T05:06:24.310ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c867bbf7-921c-4db1-887a-4002bcc6b446/crypt/zone"), full_name: "oxp_c867bbf7-921c-4db1-887a-4002bcc6b446/crypt/zone" } file = sled-storage/src/manager.rs:1331
5822024-11-13T05:06:24.320ZINFOSledAgent (ServiceManager): refreshed dendrite service with new configuration file = sled-agent/src/services.rs:4447
5832024-11-13T05:06:24.320ZINFOSledAgent (ServiceManager): configuring lldp service file = sled-agent/src/services.rs:4474
5842024-11-13T05:06:24.361ZINFOSledAgent (ServiceManager): refreshed lldpd service with new configuration file = sled-agent/src/services.rs:4487
5852024-11-13T05:06:24.361ZINFOSledAgent (ServiceManager): configuring MGS service file = sled-agent/src/services.rs:4333
5862024-11-13T05:06:24.397ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5872024-11-13T05:06:24.398ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c9fdd7dd-67c5-4597-a857-03bc9769dc87/crypt/zone"), full_name: "oxp_c9fdd7dd-67c5-4597-a857-03bc9769dc87/crypt/zone" } file = sled-storage/src/manager.rs:1331
5882024-11-13T05:06:24.417ZINFOSledAgent (ServiceManager): refreshed MGS service with new configuration file = sled-agent/src/services.rs:4375
5892024-11-13T05:06:24.417ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:4454 rack_subnet = fd00:1122:3344::
5902024-11-13T05:06:24.438ZINFOSledAgent (ServiceManager): refreshed wicketd service with new configuration file = sled-agent/src/services.rs:4465
5912024-11-13T05:06:24.438ZINFOSledAgent (ServiceManager): configuring mgd service file = sled-agent/src/services.rs:4507
5922024-11-13T05:06:24.488ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5932024-11-13T05:06:24.488ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_c867bbf7-921c-4db1-887a-4002bcc6b446/crucible" } file = sled-storage/src/manager.rs:1331
5942024-11-13T05:06:24.511ZINFOSledAgent (ServiceManager): refreshed mgd service with new configuration file = sled-agent/src/services.rs:4539
5952024-11-13T05:06:24.511ZINFOSledAgent (ServiceManager): configuring mg-ddm service file = sled-agent/src/services.rs:4545
5962024-11-13T05:06:24.602ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5972024-11-13T05:06:24.602ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/zone/oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615"), full_name: "oxp_204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/zone/oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615" } file = sled-storage/src/manager.rs:1331
5982024-11-13T05:06:24.603ZINFOSledAgent (ServiceManager): refreshed mg-ddm service with new configuration file = sled-agent/src/services.rs:4586
5992024-11-13T05:06:24.604ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:370
6002024-11-13T05:06:24.644ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:418
6012024-11-13T05:06:24.645ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/4d997277-f640-48d7-af56-6df261d5afe1) file = sled-agent/src/nexus.rs:527 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
6022024-11-13T05:06:24.686ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:686
6032024-11-13T05:06:24.686ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:731
6042024-11-13T05:06:24.686ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:734 uptime = 1
6052024-11-13T05:06:24.686ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.2.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: None,\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n tx_eq: None,\n },\n },\n },\n} file = sled-agent/src/bootstrap/early_networking.rs:440
6062024-11-13T05:06:24.721ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6072024-11-13T05:06:24.721ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/27d5a04d-ccb1-4219-851b-abc94499936f/crypt/zone/oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1"), full_name: "oxp_27d5a04d-ccb1-4219-851b-abc94499936f/crypt/zone/oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1" } file = sled-storage/src/manager.rs:1331
6082024-11-13T05:06:24.736ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
6092024-11-13T05:06:24.840ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6102024-11-13T05:06:24.840ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8e821c87-2d05-4e93-b160-49a645a177c5/crypt/zone/oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871"), full_name: "oxp_8e821c87-2d05-4e93-b160-49a645a177c5/crypt/zone/oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871" } file = sled-storage/src/manager.rs:1331
6112024-11-13T05:06:24.955ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6122024-11-13T05:06:24.955ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_e770fce0-3dd5-4678-9037-68ea781dbb30/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
6132024-11-13T05:06:25.059ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/4d997277-f640-48d7-af56-6df261d5afe1) file = sled-agent/src/probe_manager.rs:231 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
6142024-11-13T05:06:25.062ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6152024-11-13T05:06:25.062ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c867bbf7-921c-4db1-887a-4002bcc6b446/crypt/zone/oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf"), full_name: "oxp_c867bbf7-921c-4db1-887a-4002bcc6b446/crypt/zone/oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf" } file = sled-storage/src/manager.rs:1331
6162024-11-13T05:06:25.180ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6172024-11-13T05:06:25.181ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_c2ce9d45-3d62-4234-a9db-c1d5c8884adc/crucible" } file = sled-storage/src/manager.rs:1331
6182024-11-13T05:06:25.248ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 11661085320902503456 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
6192024-11-13T05:06:25.248ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 2533699652135011376 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
6202024-11-13T05:06:25.287ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6212024-11-13T05:06:25.287ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/zone/oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514"), full_name: "oxp_204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/zone/oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514" } file = sled-storage/src/manager.rs:1331
6222024-11-13T05:06:25.399ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6232024-11-13T05:06:25.399ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c867bbf7-921c-4db1-887a-4002bcc6b446/crypt/debug"), full_name: "oxp_c867bbf7-921c-4db1-887a-4002bcc6b446/crypt/debug" } file = sled-storage/src/manager.rs:1331
6242024-11-13T05:06:25.477ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6252024-11-13T05:06:25.477ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c2ce9d45-3d62-4234-a9db-c1d5c8884adc/crypt/zone/oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde"), full_name: "oxp_c2ce9d45-3d62-4234-a9db-c1d5c8884adc/crypt/zone/oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde" } file = sled-storage/src/manager.rs:1331
6262024-11-13T05:06:25.593ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6272024-11-13T05:06:25.593ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_27d5a04d-ccb1-4219-851b-abc94499936f/crucible" } file = sled-storage/src/manager.rs:1331
6282024-11-13T05:06:25.699ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6292024-11-13T05:06:25.699ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c9fdd7dd-67c5-4597-a857-03bc9769dc87/crypt/zone/oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea"), full_name: "oxp_c9fdd7dd-67c5-4597-a857-03bc9769dc87/crypt/zone/oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea" } file = sled-storage/src/manager.rs:1331
6302024-11-13T05:06:25.815ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/.omicron-datasets.json.tmp file = common/src/ledger.rs:199 request = datasets_ensure
6312024-11-13T05:06:25.816ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/.omicron-datasets.json.tmp file = common/src/ledger.rs:199 request = datasets_ensure
6322024-11-13T05:06:25.816ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:603
6332024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
6342024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
6352024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
6362024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
6372024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
6382024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
6392024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
6402024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
6412024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
6422024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
6432024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
6442024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
6452024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
6462024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
6472024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
6482024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
6492024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
6502024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
6512024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
6522024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
6532024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
6542024-11-13T05:06:25.816ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
6552024-11-13T05:06:25.816ZINFOSledAgent: datasets ensure: Updated storage file = sled-agent/src/sled_agent.rs:852 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
6562024-11-13T05:06:25.816ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 6926462 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:57489 req_id = dae8b4a3-bddc-4f93-9c92-7773f36fa43d response_code = 200 uri = /datasets
6572024-11-13T05:06:25.848ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: b62b18b5-c7f6-48f0-ab56-4213f6de7d44 (service), filesystem_pool: Some(ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: b7fab8bc-a491-4391-a446-e9d3c9193aca (service), filesystem_pool: Some(ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 105858cb-d3d5-4677-95c0-079ffa1259be (service), filesystem_pool: Some(ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:538 sled_address = [fd00:1122:3344:101::1]:12345
6582024-11-13T05:06:25.848ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62898
6592024-11-13T05:06:25.848ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b62b18b5-c7f6-48f0-ab56-4213f6de7d44, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
6602024-11-13T05:06:25.848ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b62b18b5-c7f6-48f0-ab56-4213f6de7d44, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: InternalDns }, responder: ... }
6612024-11-13T05:06:25.883ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b7fab8bc-a491-4391-a446-e9d3c9193aca, dataset_name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
6622024-11-13T05:06:25.883ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b7fab8bc-a491-4391-a446-e9d3c9193aca, dataset_name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6632024-11-13T05:06:25.917ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 105858cb-d3d5-4677-95c0-079ffa1259be, dataset_name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
6642024-11-13T05:06:25.917ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 105858cb-d3d5-4677-95c0-079ffa1259be, dataset_name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6652024-11-13T05:06:25.950ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6662024-11-13T05:06:25.950ZINFOSledAgent (ServiceManager): No ledger in /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/omicron-zones.json file = common/src/ledger.rs:188
6672024-11-13T05:06:25.950ZINFOSledAgent (ServiceManager): No ledger in /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/omicron-zones.json file = common/src/ledger.rs:188
6682024-11-13T05:06:25.950ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6692024-11-13T05:06:25.996ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6702024-11-13T05:06:25.996ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6712024-11-13T05:06:25.996ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6722024-11-13T05:06:26.054ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca file = illumos-utils/src/zone.rs:325
6732024-11-13T05:06:26.058ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be file = illumos-utils/src/zone.rs:325
6742024-11-13T05:06:26.062ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/4d997277-f640-48d7-af56-6df261d5afe1) file = sled-agent/src/probe_manager.rs:231 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
6752024-11-13T05:06:26.063ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44 file = illumos-utils/src/zone.rs:325
6762024-11-13T05:06:26.075ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca file = illumos-utils/src/zone.rs:363
6772024-11-13T05:06:26.080ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be file = illumos-utils/src/zone.rs:363
6782024-11-13T05:06:26.084ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44 file = illumos-utils/src/zone.rs:363
6792024-11-13T05:06:26.642ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/4d997277-f640-48d7-af56-6df261d5afe1) file = sled-agent/src/nexus.rs:527 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
6802024-11-13T05:06:27.064ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/4d997277-f640-48d7-af56-6df261d5afe1) file = sled-agent/src/probe_manager.rs:231 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
6812024-11-13T05:06:27.679ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService2'/> <propval type="astring" name="gateway" value='fd00:1122:3344:3::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:3::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:3::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:3::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6822024-11-13T05:06:27.679ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:3::, width: 64 }
6832024-11-13T05:06:27.679ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be
6842024-11-13T05:06:28.168ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService3'/> <propval type="astring" name="gateway" value='fd00:1122:3344:1::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:1::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:1::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:1::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6852024-11-13T05:06:28.168ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:1::, width: 64 }
6862024-11-13T05:06:28.651ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService1'/> <propval type="astring" name="gateway" value='fd00:1122:3344:2::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:2::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:2::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:2::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6872024-11-13T05:06:28.659ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be") failed: Property not found. retry in 39.163286ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be
6882024-11-13T05:06:28.659ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44
6892024-11-13T05:06:28.663ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:2::, width: 64 }
6902024-11-13T05:06:28.663ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca
6912024-11-13T05:06:28.707ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be") failed: Property not found. retry in 72.579392ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be
6922024-11-13T05:06:28.787ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44") failed: Property not found. retry in 57.740292ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44
6932024-11-13T05:06:28.794ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca") failed: Property not found. retry in 25.558243ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca
6942024-11-13T05:06:28.801ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be") failed: Property not found. retry in 179.541144ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be
6952024-11-13T05:06:28.827ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca") failed: Property not found. retry in 100.000428ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca
6962024-11-13T05:06:28.856ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44") failed: Property not found. retry in 71.232764ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44
6972024-11-13T05:06:28.947ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca") failed: Property not found. retry in 109.370348ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca
6982024-11-13T05:06:28.961ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44") failed: Property not found. retry in 141.786794ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44
6992024-11-13T05:06:29.055ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be") failed: Property not found. retry in 222.048709ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be
7002024-11-13T05:06:29.088ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca") failed: Property not found. retry in 261.697742ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca
7012024-11-13T05:06:29.119ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44") failed: Property not found. retry in 467.707169ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44
7022024-11-13T05:06:29.287ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be") failed: Property not found. retry in 641.240097ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be
7032024-11-13T05:06:29.362ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca") failed: Property not found. retry in 914.501661ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca
7042024-11-13T05:06:29.595ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44") failed: Property not found. retry in 906.408246ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44
7052024-11-13T05:06:29.938ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be") failed: Property not found. retry in 1.252095066s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be
7062024-11-13T05:06:30.285ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca") failed: Property not found. retry in 1.170397739s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca
7072024-11-13T05:06:30.512ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44") failed: Property not found. retry in 1.361679605s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44
7082024-11-13T05:06:31.199ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be") failed: Property not found. retry in 897.252412ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be
7092024-11-13T05:06:31.463ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca") failed: Property not found. retry in 1.178345039s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca
7102024-11-13T05:06:31.881ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44") failed: Property not found. retry in 1.20175268s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44
7112024-11-13T05:06:32.104ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be") failed: Property not found. retry in 1.267003632s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be
7122024-11-13T05:06:32.650ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca") failed: Property not found. retry in 695.097515ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca
7132024-11-13T05:06:33.092ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44") failed: Property not found. retry in 691.842912ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44
7142024-11-13T05:06:33.354ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca") failed: Property not found. retry in 1.182904333s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca
7152024-11-13T05:06:33.379ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be") failed: Property not found. retry in 911.687044ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be
7162024-11-13T05:06:33.921ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 5578476604298381371 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
7172024-11-13T05:06:33.921ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8090005703617564393 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
7182024-11-13T05:06:34.556ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_internal_dns_b7fab8bc-a491-4391-a446-e9d3c9193aca
7192024-11-13T05:06:34.556ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_internal_dns_105858cb-d3d5-4677-95c0-079ffa1259be
7202024-11-13T05:06:34.556ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_internal_dns_b62b18b5-c7f6-48f0-ab56-4213f6de7d44
7212024-11-13T05:06:34.556ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7222024-11-13T05:06:34.556ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7232024-11-13T05:06:34.557ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 8708735 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:62898 req_id = b35d9377-8754-49be-a68d-856fcb9fdfa4 response_code = 204 uri = /omicron-zones
7242024-11-13T05:06:34.559ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:655
7252024-11-13T05:06:34.825ZWARNSledAgent (RSS): failed to write DNS configuration (will retry in 80.03311ms) dns_config_addr = [fd00:1122:3344:1::1]:5353 error_message = Communication Error: error sending request for url (http://[fd00:1122:3344:1::1]:5353/config) file = sled-agent/src/rack_setup/service.rs:675
7262024-11-13T05:06:34.928ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:655
7272024-11-13T05:06:34.957ZWARNSledAgent (RSS): failed to write DNS configuration (will retry in 52.37384ms) dns_config_addr = [fd00:1122:3344:2::1]:5353 error_message = Communication Error: error sending request for url (http://[fd00:1122:3344:2::1]:5353/config) file = sled-agent/src/rack_setup/service.rs:675
7282024-11-13T05:06:35.036ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:655
7292024-11-13T05:06:35.098ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:691
7302024-11-13T05:06:35.098ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
7312024-11-13T05:06:35.131ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
7322024-11-13T05:06:35.132ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
7332024-11-13T05:06:35.132ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
7342024-11-13T05:06:35.161ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: b62b18b5-c7f6-48f0-ab56-4213f6de7d44 (service), filesystem_pool: Some(ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: b7fab8bc-a491-4391-a446-e9d3c9193aca (service), filesystem_pool: Some(ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 105858cb-d3d5-4677-95c0-079ffa1259be (service), filesystem_pool: Some(ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 5ed5ef92-d882-4599-99b9-5a556ad16583 (service), filesystem_pool: Some(ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 3458d198-d6c2-45f6-bb2b-84f2ec4d6bca, kind: Service { id: 5ed5ef92-d882-4599-99b9-5a556ad16583 }, name: Name("ntp-5ed5ef92-d882-4599-99b9-5a556ad16583"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 200, 81])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:538 sled_address = [fd00:1122:3344:101::1]:12345
7352024-11-13T05:06:35.161ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:53666
7362024-11-13T05:06:35.161ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b62b18b5-c7f6-48f0-ab56-4213f6de7d44, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
7372024-11-13T05:06:35.161ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b62b18b5-c7f6-48f0-ab56-4213f6de7d44, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: InternalDns }, responder: ... }
7382024-11-13T05:06:35.204ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b7fab8bc-a491-4391-a446-e9d3c9193aca, dataset_name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
7392024-11-13T05:06:35.204ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b7fab8bc-a491-4391-a446-e9d3c9193aca, dataset_name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7402024-11-13T05:06:35.242ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 105858cb-d3d5-4677-95c0-079ffa1259be, dataset_name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
7412024-11-13T05:06:35.242ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 105858cb-d3d5-4677-95c0-079ffa1259be, dataset_name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7422024-11-13T05:06:35.250ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 11661085320902503456 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
7432024-11-13T05:06:35.250ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 2533699652135011376 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
7442024-11-13T05:06:35.280ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
7452024-11-13T05:06:35.280ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/omicron-zones.json file = common/src/ledger.rs:177
7462024-11-13T05:06:35.280ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/omicron-zones.json file = common/src/ledger.rs:177
7472024-11-13T05:06:35.280ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
7482024-11-13T05:06:35.291ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
7492024-11-13T05:06:35.324ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
7502024-11-13T05:06:35.325ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
7512024-11-13T05:06:35.325ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
7522024-11-13T05:06:35.354ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte0", ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 200, 81]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 } }) }
7532024-11-13T05:06:35.354ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = boundary_ntp
7542024-11-13T05:06:35.355ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:83
7552024-11-13T05:06:35.355ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:90
7562024-11-13T05:06:35.355ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:118
7572024-11-13T05:06:35.356ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
7582024-11-13T05:06:35.379ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583 file = illumos-utils/src/zone.rs:325
7592024-11-13T05:06:35.399ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583 file = illumos-utils/src/zone.rs:363
7602024-11-13T05:06:36.056ZINFOSledAgent (ServiceManager): Profile for oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService4'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/chrony-setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="allow" value='fd00:1122:3344:100::/56'/> <propval type="boolean" name="boundary" value='true'/> <propval type="astring" name="boundary_pool" value='boundary-ntp.control-plane.oxide.internal'/> <propval type="astring" name="server" value='0.pool.ntp.org'/> </property_group> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="1.1.1.1" /> <value_node value="9.9.9.9" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/ntp"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte0'/> <propval type="astring" name="gateway" value='172.30.3.1'/> <propval type="astring" name="ip" value='172.30.3.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7612024-11-13T05:06:36.056ZINFOSledAgent (ServiceManager): Booting oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583
7622024-11-13T05:06:36.184ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583") failed: Property not found. retry in 47.276125ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583
7632024-11-13T05:06:36.238ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583") failed: Property not found. retry in 139.624926ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583
7642024-11-13T05:06:36.390ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583") failed: Property not found. retry in 150.198164ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583
7652024-11-13T05:06:36.549ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583") failed: Property not found. retry in 300.314704ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583
7662024-11-13T05:06:36.858ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583") failed: Property not found. retry in 827.229088ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583
7672024-11-13T05:06:37.696ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583") failed: Property not found. retry in 1.219785051s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583
7682024-11-13T05:06:38.926ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583") failed: Property not found. retry in 1.110959138s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583
7692024-11-13T05:06:40.047ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583") failed: Property not found. retry in 1.349500649s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583
7702024-11-13T05:06:41.420ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583
7712024-11-13T05:06:41.420ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7722024-11-13T05:06:41.420ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7732024-11-13T05:06:41.421ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 6259379 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:53666 req_id = e163f38f-727c-4ce3-b812-769ea723428b response_code = 204 uri = /omicron-zones
7742024-11-13T05:06:41.423ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:732
7752024-11-13T05:06:41.452ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
7762024-11-13T05:06:41.452ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:53828
7772024-11-13T05:06:41.461ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3900
7782024-11-13T05:06:41.461ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 8614 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:53828 req_id = 173cc32b-d9e2-4096-925d-6095e97d4524 response_code = 500 uri = /timesync
7792024-11-13T05:06:41.463ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
7802024-11-13T05:06:41.595ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
7812024-11-13T05:06:41.596ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:37204
7822024-11-13T05:06:45.604ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3900
7832024-11-13T05:06:45.605ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 4008999 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:37204 req_id = a38f510c-55eb-4c90-bc0a-310323306a94 response_code = 500 uri = /timesync
7842024-11-13T05:06:45.606ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 5784829778261408487 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
7852024-11-13T05:06:45.607ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 5578476604298381371 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
7862024-11-13T05:06:45.607ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8090005703617564393 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
7872024-11-13T05:06:45.607ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 3991179191140788418 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
7882024-11-13T05:06:45.607ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 4911092712773259668 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
7892024-11-13T05:06:45.607ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 11661085320902503456 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
7902024-11-13T05:06:45.607ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 2533699652135011376 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
7912024-11-13T05:06:45.608ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
7922024-11-13T05:06:45.741ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
7932024-11-13T05:06:45.741ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:58138
7942024-11-13T05:06:45.749ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3900
7952024-11-13T05:06:45.749ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 8578 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:58138 req_id = f2818fc9-07b8-4507-b5de-7c0be307dac0 response_code = 500 uri = /timesync
7962024-11-13T05:06:45.751ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
7972024-11-13T05:06:45.953ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
7982024-11-13T05:06:45.953ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64403
7992024-11-13T05:06:45.961ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_5ed5ef92-d882-4599-99b9-5a556ad16583': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3900
8002024-11-13T05:06:45.961ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 8017 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:64403 req_id = bde4830b-559f-447f-907c-514056ff2427 response_code = 500 uri = /timesync
8012024-11-13T05:06:45.964ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8022024-11-13T05:06:46.124ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8032024-11-13T05:06:46.125ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:46764
8042024-11-13T05:06:48.134ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2009497 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:46764 req_id = 37fcf0e3-3938-418c-873e-470ee2596bd3 response_code = 200 uri = /timesync
8052024-11-13T05:06:48.137ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:740
8062024-11-13T05:06:48.137ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8072024-11-13T05:06:48.399ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8082024-11-13T05:06:48.399ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:41160
8092024-11-13T05:06:48.408ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 8455 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:41160 req_id = c779ccaa-ce2f-4225-9228-c419d3e841f6 response_code = 200 uri = /timesync
8102024-11-13T05:06:48.410ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:740
8112024-11-13T05:06:48.410ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8122024-11-13T05:06:48.631ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8132024-11-13T05:06:48.631ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:58524
8142024-11-13T05:06:48.639ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 8156 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:58524 req_id = ce8c6dbf-fd0c-49e6-b072-cad19a6e934c response_code = 200 uri = /timesync
8152024-11-13T05:06:48.642ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:740
8162024-11-13T05:06:48.642ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8172024-11-13T05:06:48.295ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8182024-11-13T05:06:48.295ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:52607
8192024-11-13T05:06:48.303ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 7564 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:52607 req_id = 54fb20eb-b931-4862-87be-e12598e641fb response_code = 200 uri = /timesync
8202024-11-13T05:06:48.306ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:740
8212024-11-13T05:06:48.306ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8222024-11-13T05:06:48.756ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8232024-11-13T05:06:48.757ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:59321
8242024-11-13T05:06:48.766ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 9293 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:59321 req_id = 20ca42cb-8991-4914-92f6-a065f070fe54 response_code = 200 uri = /timesync
8252024-11-13T05:06:48.768ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.000516243 } file = sled-agent/src/rack_setup/service.rs:740
8262024-11-13T05:06:48.768ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8272024-11-13T05:06:49.152ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8282024-11-13T05:06:49.152ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:43427
8292024-11-13T05:06:49.161ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 8744 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:43427 req_id = 84c6e99a-336d-452a-bc9d-cf2df56067ba response_code = 200 uri = /timesync
8302024-11-13T05:06:49.164ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.000239634 } file = sled-agent/src/rack_setup/service.rs:740
8312024-11-13T05:06:49.164ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8322024-11-13T05:06:49.693ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8332024-11-13T05:06:49.694ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:56392
8342024-11-13T05:06:49.703ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 8987 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:56392 req_id = cba3c888-214b-4246-92b1-37d138831ae6 response_code = 200 uri = /timesync
8352024-11-13T05:06:49.706ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.000381498 } file = sled-agent/src/rack_setup/service.rs:740
8362024-11-13T05:06:49.706ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8372024-11-13T05:06:50.075ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8382024-11-13T05:06:50.075ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:49700
8392024-11-13T05:06:50.084ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 8836 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:49700 req_id = 7a2203c2-42c1-43ef-a241-0a7400a2c231 response_code = 200 uri = /timesync
8402024-11-13T05:06:50.088ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.000201838 } file = sled-agent/src/rack_setup/service.rs:740
8412024-11-13T05:06:50.088ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:762
8422024-11-13T05:06:50.624ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 14920098166117133754 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
8432024-11-13T05:06:50.624ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 3964936605692988581 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
8442024-11-13T05:06:50.682ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:712
8452024-11-13T05:06:50.682ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:40922
8462024-11-13T05:06:50.703ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 21315 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:40922 req_id = 5978a0d2-abf8-4bc2-b1e2-b75e85ed612a response_code = 200 uri = /timesync
8472024-11-13T05:06:50.708ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 1753455535, ip_addr: 104.131.155.175, stratum: 3, ref_time: 1731474410.1714334, correction: 1.3487e-5 } file = sled-agent/src/rack_setup/service.rs:740
8482024-11-13T05:06:50.708ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1423
8492024-11-13T05:06:50.738ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: b62b18b5-c7f6-48f0-ab56-4213f6de7d44 (service), filesystem_pool: Some(ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: b7fab8bc-a491-4391-a446-e9d3c9193aca (service), filesystem_pool: Some(ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 105858cb-d3d5-4677-95c0-079ffa1259be (service), filesystem_pool: Some(ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 5ed5ef92-d882-4599-99b9-5a556ad16583 (service), filesystem_pool: Some(ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 3458d198-d6c2-45f6-bb2b-84f2ec4d6bca, kind: Service { id: 5ed5ef92-d882-4599-99b9-5a556ad16583 }, name: Name("ntp-5ed5ef92-d882-4599-99b9-5a556ad16583"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 200, 81])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: ac51e8a5-e331-4714-98dc-b892a6790474 (service), filesystem_pool: Some(ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External } } } }, OmicronZoneConfig { id: ab2166ea-ef2a-4d3c-97f0-caf366d719f6 (service), filesystem_pool: Some(ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External } } } }, OmicronZoneConfig { id: f777a96f-c504-429f-9974-554a2d6f2615 (service), filesystem_pool: Some(ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External } } } }, OmicronZoneConfig { id: 8e52b1f1-1193-4ae0-9fcf-b47645507525 (service), filesystem_pool: Some(ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External } } } }, OmicronZoneConfig { id: e942c185-9d45-41ec-a5c5-95481947c4ca (service), filesystem_pool: Some(ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:538 sled_address = [fd00:1122:3344:101::1]:12345
8502024-11-13T05:06:50.738ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:61487
8512024-11-13T05:06:50.739ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b62b18b5-c7f6-48f0-ab56-4213f6de7d44, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8522024-11-13T05:06:50.739ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b62b18b5-c7f6-48f0-ab56-4213f6de7d44, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: InternalDns }, responder: ... }
8532024-11-13T05:06:50.775ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b7fab8bc-a491-4391-a446-e9d3c9193aca, dataset_name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8542024-11-13T05:06:50.775ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b7fab8bc-a491-4391-a446-e9d3c9193aca, dataset_name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8552024-11-13T05:06:50.810ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 105858cb-d3d5-4677-95c0-079ffa1259be, dataset_name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8562024-11-13T05:06:50.810ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 105858cb-d3d5-4677-95c0-079ffa1259be, dataset_name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8572024-11-13T05:06:50.843ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ac51e8a5-e331-4714-98dc-b892a6790474, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8582024-11-13T05:06:50.843ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ac51e8a5-e331-4714-98dc-b892a6790474, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: Cockroach }, responder: ... }
8592024-11-13T05:06:50.878ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ab2166ea-ef2a-4d3c-97f0-caf366d719f6, dataset_name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8602024-11-13T05:06:50.878ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ab2166ea-ef2a-4d3c-97f0-caf366d719f6, dataset_name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8612024-11-13T05:06:50.912ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f777a96f-c504-429f-9974-554a2d6f2615, dataset_name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8622024-11-13T05:06:50.912ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: f777a96f-c504-429f-9974-554a2d6f2615, dataset_name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8632024-11-13T05:06:50.944ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 8e52b1f1-1193-4ae0-9fcf-b47645507525, dataset_name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8642024-11-13T05:06:50.944ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 8e52b1f1-1193-4ae0-9fcf-b47645507525, dataset_name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8652024-11-13T05:06:50.979ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e942c185-9d45-41ec-a5c5-95481947c4ca, dataset_name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8662024-11-13T05:06:50.979ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: e942c185-9d45-41ec-a5c5-95481947c4ca, dataset_name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8672024-11-13T05:06:51.014ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8682024-11-13T05:06:51.014ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/omicron-zones.json file = common/src/ledger.rs:177
8692024-11-13T05:06:51.015ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/omicron-zones.json file = common/src/ledger.rs:177
8702024-11-13T05:06:51.015ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8712024-11-13T05:06:51.094ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8722024-11-13T05:06:51.094ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8732024-11-13T05:06:51.094ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8742024-11-13T05:06:51.094ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8752024-11-13T05:06:51.094ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8762024-11-13T05:06:51.188ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474 file = illumos-utils/src/zone.rs:325
8772024-11-13T05:06:51.193ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615 file = illumos-utils/src/zone.rs:325
8782024-11-13T05:06:51.197ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525 file = illumos-utils/src/zone.rs:325
8792024-11-13T05:06:51.202ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca file = illumos-utils/src/zone.rs:325
8802024-11-13T05:06:51.206ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6 file = illumos-utils/src/zone.rs:325
8812024-11-13T05:06:51.210ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474 file = illumos-utils/src/zone.rs:363
8822024-11-13T05:06:51.215ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615 file = illumos-utils/src/zone.rs:363
8832024-11-13T05:06:51.220ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525 file = illumos-utils/src/zone.rs:363
8842024-11-13T05:06:51.224ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca file = illumos-utils/src/zone.rs:363
8852024-11-13T05:06:51.228ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6 file = illumos-utils/src/zone.rs:363
8862024-11-13T05:06:53.569ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47397
8872024-11-13T05:06:53.569ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 88 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47397 req_id = 1a447786-0b6a-4c7f-8e6c-9234dcad4d13 response_code = 200 uri = /baseboard
8882024-11-13T05:06:54.147ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService5'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::3'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='ac51e8a5-e331-4714-98dc-b892a6790474'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::3]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8892024-11-13T05:06:54.147ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService7'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='8e52b1f1-1193-4ae0-9fcf-b47645507525'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::6]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8902024-11-13T05:06:54.147ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474
8912024-11-13T05:06:54.153ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525
8922024-11-13T05:06:54.178ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService9'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::4'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='ab2166ea-ef2a-4d3c-97f0-caf366d719f6'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::4]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8932024-11-13T05:06:54.178ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6
8942024-11-13T05:06:54.216ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService6'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='f777a96f-c504-429f-9974-554a2d6f2615'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::5]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8952024-11-13T05:06:54.216ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615
8962024-11-13T05:06:54.228ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService8'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='e942c185-9d45-41ec-a5c5-95481947c4ca'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::7]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8972024-11-13T05:06:54.228ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca
8982024-11-13T05:06:54.296ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474") failed: Property not found. retry in 25.931841ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474
8992024-11-13T05:06:54.304ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525") failed: Property not found. retry in 66.371233ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525
9002024-11-13T05:06:54.318ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6") failed: Property not found. retry in 45.158647ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6
9012024-11-13T05:06:54.331ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474") failed: Property not found. retry in 75.238441ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474
9022024-11-13T05:06:54.347ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615") failed: Property not found. retry in 45.391435ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615
9032024-11-13T05:06:55.216ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6") failed: Property not found. retry in 114.746358ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6
9042024-11-13T05:06:55.572ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474") failed: Property not found. retry in 288.6498ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474
9052024-11-13T05:06:57.257ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615") failed: Property not found. retry in 96.455429ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615
9062024-11-13T05:06:57.377ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525") failed: Property not found. retry in 107.322695ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525
9072024-11-13T05:06:57.414ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca") failed: Property not found. retry in 69.482105ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca
9082024-11-13T05:06:57.425ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6") failed: Property not found. retry in 164.881605ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6
9092024-11-13T05:06:57.435ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474") failed: Property not found. retry in 508.434498ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474
9102024-11-13T05:06:57.443ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615") failed: Property not found. retry in 170.528152ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615
9112024-11-13T05:06:57.494ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525") failed: Property not found. retry in 203.901146ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525
9122024-11-13T05:06:57.502ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca") failed: Property not found. retry in 91.623211ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca
9132024-11-13T05:06:57.601ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6") failed: Property not found. retry in 453.439752ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6
9142024-11-13T05:06:57.610ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca") failed: Property not found. retry in 293.913367ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca
9152024-11-13T05:06:57.621ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615") failed: Property not found. retry in 559.696826ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615
9162024-11-13T05:06:57.708ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525") failed: Property not found. retry in 338.961559ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525
9172024-11-13T05:06:57.916ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca") failed: Property not found. retry in 580.732869ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca
9182024-11-13T05:06:57.961ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474") failed: Property not found. retry in 902.481606ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474
9192024-11-13T05:06:58.058ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525") failed: Property not found. retry in 1.168099744s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525
9202024-11-13T05:06:58.067ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6") failed: Property not found. retry in 672.517289ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6
9212024-11-13T05:06:58.192ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615") failed: Property not found. retry in 552.95203ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615
9222024-11-13T05:06:58.508ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca") failed: Property not found. retry in 553.291639ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca
9232024-11-13T05:06:58.751ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6") failed: Property not found. retry in 1.127405551s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6
9242024-11-13T05:06:58.758ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615") failed: Property not found. retry in 1.326278049s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615
9252024-11-13T05:06:58.873ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474") failed: Property not found. retry in 1.110867535s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474
9262024-11-13T05:06:59.071ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca") failed: Property not found. retry in 876.43652ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca
9272024-11-13T05:06:59.236ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525") failed: Property not found. retry in 1.358992729s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525
9282024-11-13T05:06:59.843ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 305.346459ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:424 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
9292024-11-13T05:06:59.888ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6") failed: Property not found. retry in 784.766303ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6
9302024-11-13T05:06:59.958ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca") failed: Property not found. retry in 1.0700193s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca
9312024-11-13T05:06:59.993ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474") failed: Property not found. retry in 1.485014379s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474
9322024-11-13T05:07:00.094ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615") failed: Property not found. retry in 1.067810374s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615
9332024-11-13T05:07:00.606ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525") failed: Property not found. retry in 978.168071ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525
9342024-11-13T05:07:00.683ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6") failed: Property not found. retry in 1.184911841s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6
9352024-11-13T05:07:01.039ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca") failed: Property not found. retry in 1.157696736s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca
9362024-11-13T05:07:01.171ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615") failed: Property not found. retry in 928.017381ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615
9372024-11-13T05:07:01.487ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474") failed: Property not found. retry in 1.285609352s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474
9382024-11-13T05:07:01.594ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525") failed: Property not found. retry in 1.452165536s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525
9392024-11-13T05:07:01.877ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6") failed: Property not found. retry in 539.916456ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6
9402024-11-13T05:07:02.107ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615") failed: Property not found. retry in 1.303149354s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615
9412024-11-13T05:07:02.206ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca") failed: Property not found. retry in 979.471004ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca
9422024-11-13T05:07:02.427ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6") failed: Property not found. retry in 699.649222ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6
9432024-11-13T05:07:02.781ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474") failed: Property not found. retry in 1.29762921s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474
9442024-11-13T05:07:03.055ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525") failed: Property not found. retry in 1.478561907s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525
9452024-11-13T05:07:03.135ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6") failed: Property not found. retry in 1.161461618s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6
9462024-11-13T05:07:03.193ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca") failed: Property not found. retry in 817.010407ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca
9472024-11-13T05:07:03.420ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615") failed: Property not found. retry in 1.410568872s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615
9482024-11-13T05:07:04.021ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca") failed: Property not found. retry in 1.170230116s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca
9492024-11-13T05:07:04.088ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474") failed: Property not found. retry in 1.025563012s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474
9502024-11-13T05:07:04.307ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6") failed: Property not found. retry in 579.24475ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6
9512024-11-13T05:07:04.543ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525") failed: Property not found. retry in 541.506671ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525
9522024-11-13T05:07:04.840ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615") failed: Property not found. retry in 833.934531ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615
9532024-11-13T05:07:04.895ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6") failed: Property not found. retry in 714.525294ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6
9542024-11-13T05:07:05.093ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525") failed: Property not found. retry in 1.013106241s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525
9552024-11-13T05:07:05.123ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474") failed: Property not found. retry in 1.430370164s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474
9562024-11-13T05:07:05.204ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca") failed: Property not found. retry in 593.402222ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca
9572024-11-13T05:07:05.619ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6") failed: Property not found. retry in 940.462875ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6
9582024-11-13T05:07:05.684ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615") failed: Property not found. retry in 1.430797172s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615
9592024-11-13T05:07:05.807ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca") failed: Property not found. retry in 1.323460406s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca
9602024-11-13T05:07:07.156ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_cockroachdb_ac51e8a5-e331-4714-98dc-b892a6790474
9612024-11-13T05:07:07.156ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_cockroachdb_f777a96f-c504-429f-9974-554a2d6f2615
9622024-11-13T05:07:07.156ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525
9632024-11-13T05:07:07.156ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_cockroachdb_e942c185-9d45-41ec-a5c5-95481947c4ca
9642024-11-13T05:07:07.156ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_cockroachdb_ab2166ea-ef2a-4d3c-97f0-caf366d719f6
9652024-11-13T05:07:07.156ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
9662024-11-13T05:07:07.156ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
9672024-11-13T05:07:07.156ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 16417384 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:61487 req_id = 36de1e1d-17b0-4d52-9da9-e3668342cdc4 response_code = 204 uri = /omicron-zones
9682024-11-13T05:07:07.197ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:54824
9692024-11-13T05:07:07.227ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService7/omicron6 in oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService7/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 30014 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:54824 req_id = 79e3aacb-fc9d-4bcc-94ba-2bcca410c593 response_code = 500 uri = /cockroachdb
9702024-11-13T05:07:07.227ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "79e3aacb-fc9d-4bcc-94ba-2bcca410c593", "content-length": "124", "date": "Wed, 13 Nov 2024 05:07:07 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "79e3aacb-fc9d-4bcc-94ba-2bcca410c593" } file = sled-agent/src/rack_setup/service.rs:1152 retry_after = 53.494563ms
9712024-11-13T05:07:07.309ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService7/omicron6 in oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService7/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 27201 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:54824 req_id = f0d991c4-f3d6-4ee0-8832-94eb3d92d229 response_code = 500 uri = /cockroachdb
9722024-11-13T05:07:07.310ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "f0d991c4-f3d6-4ee0-8832-94eb3d92d229", "content-length": "124", "date": "Wed, 13 Nov 2024 05:07:07 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "f0d991c4-f3d6-4ee0-8832-94eb3d92d229" } file = sled-agent/src/rack_setup/service.rs:1152 retry_after = 68.259418ms
9732024-11-13T05:07:07.405ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService7/omicron6 in oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService7/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 25954 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:54824 req_id = 777eec3c-f683-4d80-bb54-640319cc7a74 response_code = 500 uri = /cockroachdb
9742024-11-13T05:07:07.405ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "777eec3c-f683-4d80-bb54-640319cc7a74", "content-length": "124", "date": "Wed, 13 Nov 2024 05:07:07 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "777eec3c-f683-4d80-bb54-640319cc7a74" } file = sled-agent/src/rack_setup/service.rs:1152 retry_after = 154.612974ms
9752024-11-13T05:07:07.588ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService7/omicron6 in oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService7/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 26833 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:54824 req_id = 6b0c0e5c-247d-4ecd-960e-fdb2c8c876db response_code = 500 uri = /cockroachdb
9762024-11-13T05:07:07.588ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "6b0c0e5c-247d-4ecd-960e-fdb2c8c876db", "content-length": "124", "date": "Wed, 13 Nov 2024 05:07:07 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "6b0c0e5c-247d-4ecd-960e-fdb2c8c876db" } file = sled-agent/src/rack_setup/service.rs:1152 retry_after = 231.22007ms
9772024-11-13T05:07:07.847ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService7/omicron6 in oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService7/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 26505 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:54824 req_id = f2e72e07-4623-44f8-8795-86c3be09ea8c response_code = 500 uri = /cockroachdb
9782024-11-13T05:07:07.847ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "f2e72e07-4623-44f8-8795-86c3be09ea8c", "content-length": "124", "date": "Wed, 13 Nov 2024 05:07:07 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "f2e72e07-4623-44f8-8795-86c3be09ea8c" } file = sled-agent/src/rack_setup/service.rs:1152 retry_after = 243.40573ms
9792024-11-13T05:07:08.117ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService7/omicron6 in oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService7/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 25707 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:54824 req_id = 999a2434-e32e-425d-a06a-2ecff1de92f4 response_code = 500 uri = /cockroachdb
9802024-11-13T05:07:08.117ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "999a2434-e32e-425d-a06a-2ecff1de92f4", "content-length": "124", "date": "Wed, 13 Nov 2024 05:07:07 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "999a2434-e32e-425d-a06a-2ecff1de92f4" } file = sled-agent/src/rack_setup/service.rs:1152 retry_after = 269.775714ms
9812024-11-13T05:07:08.413ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService7/omicron6 in oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService7/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 25521 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:54824 req_id = 6a9cdc79-b48c-4474-a9f8-9ec3ccd155f1 response_code = 500 uri = /cockroachdb
9822024-11-13T05:07:08.413ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "6a9cdc79-b48c-4474-a9f8-9ec3ccd155f1", "content-length": "124", "date": "Wed, 13 Nov 2024 05:07:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "6a9cdc79-b48c-4474-a9f8-9ec3ccd155f1" } file = sled-agent/src/rack_setup/service.rs:1152 retry_after = 234.091068ms
9832024-11-13T05:07:08.674ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService7/omicron6 in oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService7/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 26042 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:54824 req_id = e1973533-f4a9-47d7-a6b6-d4942ae7959f response_code = 500 uri = /cockroachdb
9842024-11-13T05:07:08.674ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "e1973533-f4a9-47d7-a6b6-d4942ae7959f", "content-length": "124", "date": "Wed, 13 Nov 2024 05:07:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "e1973533-f4a9-47d7-a6b6-d4942ae7959f" } file = sled-agent/src/rack_setup/service.rs:1152 retry_after = 379.073625ms
9852024-11-13T05:07:09.080ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService7/omicron6 in oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService7/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 25574 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:54824 req_id = d75c13a7-cf07-4464-a664-0baca3f496a1 response_code = 500 uri = /cockroachdb
9862024-11-13T05:07:09.081ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "d75c13a7-cf07-4464-a664-0baca3f496a1", "content-length": "124", "date": "Wed, 13 Nov 2024 05:07:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "d75c13a7-cf07-4464-a664-0baca3f496a1" } file = sled-agent/src/rack_setup/service.rs:1152 retry_after = 388.866993ms
9872024-11-13T05:07:09.501ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService7/omicron6 in oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_8e52b1f1-1193-4ae0-9fcf-b47645507525 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService7/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:938 latency_us = 30011 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:54824 req_id = 388ae6bd-2967-4781-bc99-d810c13a9d21 response_code = 500 uri = /cockroachdb
9882024-11-13T05:07:09.501ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "388ae6bd-2967-4781-bc99-d810c13a9d21", "content-length": "124", "date": "Wed, 13 Nov 2024 05:07:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "388ae6bd-2967-4781-bc99-d810c13a9d21" } file = sled-agent/src/rack_setup/service.rs:1152 retry_after = 618.933653ms
9892024-11-13T05:07:10.153ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::6]:32221 file = sled-agent/src/services.rs:3743
9902024-11-13T05:07:19.662ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3761
9912024-11-13T05:07:22.949ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:3784
9922024-11-13T05:07:22.949ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 12827224 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:54824 req_id = e2e2fe33-ba00-4954-ad91-ba4102713cbd response_code = 204 uri = /cockroachdb
9932024-11-13T05:07:22.954ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 695.045254ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:424 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
9942024-11-13T05:07:22.987ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: b62b18b5-c7f6-48f0-ab56-4213f6de7d44 (service), filesystem_pool: Some(ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: b7fab8bc-a491-4391-a446-e9d3c9193aca (service), filesystem_pool: Some(ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 105858cb-d3d5-4677-95c0-079ffa1259be (service), filesystem_pool: Some(ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 5ed5ef92-d882-4599-99b9-5a556ad16583 (service), filesystem_pool: Some(ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 3458d198-d6c2-45f6-bb2b-84f2ec4d6bca, kind: Service { id: 5ed5ef92-d882-4599-99b9-5a556ad16583 }, name: Name("ntp-5ed5ef92-d882-4599-99b9-5a556ad16583"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 200, 81])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: ac51e8a5-e331-4714-98dc-b892a6790474 (service), filesystem_pool: Some(ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External } } } }, OmicronZoneConfig { id: ab2166ea-ef2a-4d3c-97f0-caf366d719f6 (service), filesystem_pool: Some(ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External } } } }, OmicronZoneConfig { id: f777a96f-c504-429f-9974-554a2d6f2615 (service), filesystem_pool: Some(ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External } } } }, OmicronZoneConfig { id: 8e52b1f1-1193-4ae0-9fcf-b47645507525 (service), filesystem_pool: Some(ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External } } } }, OmicronZoneConfig { id: e942c185-9d45-41ec-a5c5-95481947c4ca (service), filesystem_pool: Some(ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External } } } }, OmicronZoneConfig { id: eb4a98a4-3147-445f-9881-33f4dffeca66 (service), filesystem_pool: Some(ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.2.164:53, nic: NetworkInterface { id: b98af94a-ceb1-41e9-82a5-9be8a66b4c85, kind: Service { id: eb4a98a4-3147-445f-9881-33f4dffeca66 }, name: Name("external-dns-eb4a98a4-3147-445f-9881-33f4dffeca66"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 145])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: 038e4c2b-2408-42f1-b025-c88f1167faaf (service), filesystem_pool: Some(ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.2.165:53, nic: NetworkInterface { id: 150e1804-b3f0-4f2c-82cf-fb2bbaab0984, kind: Service { id: 038e4c2b-2408-42f1-b025-c88f1167faaf }, name: Name("external-dns-038e4c2b-2408-42f1-b025-c88f1167faaf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 226, 83])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: e181617d-520a-429d-85ef-385f6dde38ea (service), filesystem_pool: Some(ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.2.166, nic: NetworkInterface { id: 984220cf-9c9a-412c-a465-3579565ca87a, kind: Service { id: e181617d-520a-429d-85ef-385f6dde38ea }, name: Name("nexus-e181617d-520a-429d-85ef-385f6dde38ea"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 83])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 7641195e-73b4-4972-ad51-4fcacb36a4be (service), filesystem_pool: Some(ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.2.167, nic: NetworkInterface { id: 55f88ed5-95d7-43db-b06a-514b7adb4cdb, kind: Service { id: 7641195e-73b4-4972-ad51-4fcacb36a4be }, name: Name("nexus-7641195e-73b4-4972-ad51-4fcacb36a4be"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 204])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 6c4893ca-1b18-4910-8b18-504e5fcb8dc5 (service), filesystem_pool: Some(ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.2.168, nic: NetworkInterface { id: 9df31eec-63bd-49fd-99e9-b3a05e952553, kind: Service { id: 6c4893ca-1b18-4910-8b18-504e5fcb8dc5 }, name: Name("nexus-6c4893ca-1b18-4910-8b18-504e5fcb8dc5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 5])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 671c6402-97d6-48bd-9991-5554bd96a514 (service), filesystem_pool: Some(ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }), zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: d21b9c90-434a-419c-aa82-907917c5d09c (service), filesystem_pool: Some(ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }), zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External } } } }, OmicronZoneConfig { id: dfbd3921-57ae-428e-b0a3-4f8aad3d8641 (service), filesystem_pool: Some(ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: 98c5824c-c3f0-4916-bdb4-2d59cf1e6148 (service), filesystem_pool: Some(ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: 42070343-2876-47fd-85a9-8ad8c72fa237 (service), filesystem_pool: Some(ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: 4134cecf-46e8-4040-95d2-8570b8a2f208 (service), filesystem_pool: Some(ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External } } } }, OmicronZoneConfig { id: 02279d36-b72f-47ae-859e-12e5a6192317 (service), filesystem_pool: Some(ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External } } } }, OmicronZoneConfig { id: ed7f3aff-8a79-4a12-a77f-6a877e7ba03a (service), filesystem_pool: Some(ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External } } } }, OmicronZoneConfig { id: a9fdd6bc-028a-4967-ae6c-005966661871 (service), filesystem_pool: Some(ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External } } } }, OmicronZoneConfig { id: d3ea4bae-2501-4144-a858-606f20fb94c5 (service), filesystem_pool: Some(ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External } } } }, OmicronZoneConfig { id: c131db8a-f8d3-4b52-9eef-d815a35dcdde (service), filesystem_pool: Some(ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External } } } }, OmicronZoneConfig { id: 51d35f2d-6f54-4059-b52e-080f645ee9eb (service), filesystem_pool: Some(ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External } } } }, OmicronZoneConfig { id: f8c37587-dd71-4b4c-84d0-02d3d83b3ff1 (service), filesystem_pool: Some(ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External } } } }, OmicronZoneConfig { id: cb2d36d7-52c5-4aa0-bc43-f9e463290a5f (service), filesystem_pool: Some(ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:538 sled_address = [fd00:1122:3344:101::1]:12345
9952024-11-13T05:07:22.987ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:51011
9962024-11-13T05:07:22.988ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b62b18b5-c7f6-48f0-ab56-4213f6de7d44, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9972024-11-13T05:07:22.988ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b62b18b5-c7f6-48f0-ab56-4213f6de7d44, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: InternalDns }, responder: ... }
9982024-11-13T05:07:23.086ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b7fab8bc-a491-4391-a446-e9d3c9193aca, dataset_name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9992024-11-13T05:07:23.086ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b7fab8bc-a491-4391-a446-e9d3c9193aca, dataset_name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10002024-11-13T05:07:23.135ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 105858cb-d3d5-4677-95c0-079ffa1259be, dataset_name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10012024-11-13T05:07:23.135ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 105858cb-d3d5-4677-95c0-079ffa1259be, dataset_name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10022024-11-13T05:07:23.178ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ac51e8a5-e331-4714-98dc-b892a6790474, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10032024-11-13T05:07:23.178ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ac51e8a5-e331-4714-98dc-b892a6790474, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: Cockroach }, responder: ... }
10042024-11-13T05:07:23.219ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ab2166ea-ef2a-4d3c-97f0-caf366d719f6, dataset_name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10052024-11-13T05:07:23.219ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ab2166ea-ef2a-4d3c-97f0-caf366d719f6, dataset_name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10062024-11-13T05:07:23.264ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f777a96f-c504-429f-9974-554a2d6f2615, dataset_name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10072024-11-13T05:07:23.264ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: f777a96f-c504-429f-9974-554a2d6f2615, dataset_name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10082024-11-13T05:07:23.303ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 8e52b1f1-1193-4ae0-9fcf-b47645507525, dataset_name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10092024-11-13T05:07:23.303ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 8e52b1f1-1193-4ae0-9fcf-b47645507525, dataset_name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10102024-11-13T05:07:23.350ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: e942c185-9d45-41ec-a5c5-95481947c4ca, dataset_name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10112024-11-13T05:07:23.350ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: e942c185-9d45-41ec-a5c5-95481947c4ca, dataset_name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10122024-11-13T05:07:23.391ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: eb4a98a4-3147-445f-9881-33f4dffeca66, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10132024-11-13T05:07:23.391ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: eb4a98a4-3147-445f-9881-33f4dffeca66, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: ExternalDns }, responder: ... }
10142024-11-13T05:07:23.434ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 038e4c2b-2408-42f1-b025-c88f1167faaf, dataset_name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10152024-11-13T05:07:23.434ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 038e4c2b-2408-42f1-b025-c88f1167faaf, dataset_name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
10162024-11-13T05:07:23.477ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d21b9c90-434a-419c-aa82-907917c5d09c, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: Clickhouse }, responder: ... }) file = sled-storage/src/manager.rs:603
10172024-11-13T05:07:23.477ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: d21b9c90-434a-419c-aa82-907917c5d09c, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: Clickhouse }, responder: ... }
10182024-11-13T05:07:23.524ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 4134cecf-46e8-4040-95d2-8570b8a2f208, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10192024-11-13T05:07:23.524ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 4134cecf-46e8-4040-95d2-8570b8a2f208, dataset_name: DatasetName { pool_name: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, kind: Crucible }, responder: ... }
10202024-11-13T05:07:23.565ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 02279d36-b72f-47ae-859e-12e5a6192317, dataset_name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10212024-11-13T05:07:23.566ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 02279d36-b72f-47ae-859e-12e5a6192317, dataset_name: DatasetName { pool_name: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, kind: Crucible }, responder: ... }
10222024-11-13T05:07:23.607ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:56385
10232024-11-13T05:07:23.608ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 74 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:56385 req_id = 241d15f9-0200-441c-87c8-466370dd424c response_code = 200 uri = /baseboard
10242024-11-13T05:07:23.613ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ed7f3aff-8a79-4a12-a77f-6a877e7ba03a, dataset_name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10252024-11-13T05:07:23.613ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ed7f3aff-8a79-4a12-a77f-6a877e7ba03a, dataset_name: DatasetName { pool_name: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, kind: Crucible }, responder: ... }
10262024-11-13T05:07:23.656ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: a9fdd6bc-028a-4967-ae6c-005966661871, dataset_name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10272024-11-13T05:07:23.656ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: a9fdd6bc-028a-4967-ae6c-005966661871, dataset_name: DatasetName { pool_name: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, kind: Crucible }, responder: ... }
10282024-11-13T05:07:23.700ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d3ea4bae-2501-4144-a858-606f20fb94c5, dataset_name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10292024-11-13T05:07:23.700ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: d3ea4bae-2501-4144-a858-606f20fb94c5, dataset_name: DatasetName { pool_name: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, kind: Crucible }, responder: ... }
10302024-11-13T05:07:23.742ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: c131db8a-f8d3-4b52-9eef-d815a35dcdde, dataset_name: DatasetName { pool_name: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10312024-11-13T05:07:23.742ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: c131db8a-f8d3-4b52-9eef-d815a35dcdde, dataset_name: DatasetName { pool_name: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, kind: Crucible }, responder: ... }
10322024-11-13T05:07:23.782ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 51d35f2d-6f54-4059-b52e-080f645ee9eb, dataset_name: DatasetName { pool_name: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10332024-11-13T05:07:23.782ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 51d35f2d-6f54-4059-b52e-080f645ee9eb, dataset_name: DatasetName { pool_name: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, kind: Crucible }, responder: ... }
10342024-11-13T05:07:23.831ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: f8c37587-dd71-4b4c-84d0-02d3d83b3ff1, dataset_name: DatasetName { pool_name: ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10352024-11-13T05:07:23.831ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: f8c37587-dd71-4b4c-84d0-02d3d83b3ff1, dataset_name: DatasetName { pool_name: ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External }, kind: Crucible }, responder: ... }
10362024-11-13T05:07:23.879ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: cb2d36d7-52c5-4aa0-bc43-f9e463290a5f, dataset_name: DatasetName { pool_name: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
10372024-11-13T05:07:23.879ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: cb2d36d7-52c5-4aa0-bc43-f9e463290a5f, dataset_name: DatasetName { pool_name: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }, kind: Crucible }, responder: ... }
10382024-11-13T05:07:23.925ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10392024-11-13T05:07:23.925ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/omicron-zones.json file = common/src/ledger.rs:177
10402024-11-13T05:07:23.925ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/omicron-zones.json file = common/src/ledger.rs:177
10412024-11-13T05:07:23.926ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10422024-11-13T05:07:24.092ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
10432024-11-13T05:07:24.093ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10442024-11-13T05:07:24.093ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10452024-11-13T05:07:24.093ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10462024-11-13T05:07:24.093ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10472024-11-13T05:07:24.093ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10482024-11-13T05:07:24.093ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10492024-11-13T05:07:24.093ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10502024-11-13T05:07:24.093ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10512024-11-13T05:07:24.094ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
10522024-11-13T05:07:24.272ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
10532024-11-13T05:07:24.273ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10542024-11-13T05:07:24.274ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
10552024-11-13T05:07:24.323ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
10562024-11-13T05:07:24.324ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10572024-11-13T05:07:24.324ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10582024-11-13T05:07:24.324ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10592024-11-13T05:07:24.337ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10602024-11-13T05:07:24.337ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10612024-11-13T05:07:24.337ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1 file = illumos-utils/src/zone.rs:325
10622024-11-13T05:07:24.341ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5 file = illumos-utils/src/zone.rs:325
10632024-11-13T05:07:24.346ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514 file = illumos-utils/src/zone.rs:325
10642024-11-13T05:07:24.351ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c file = illumos-utils/src/zone.rs:325
10652024-11-13T05:07:24.356ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a file = illumos-utils/src/zone.rs:325
10662024-11-13T05:07:24.361ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148 file = illumos-utils/src/zone.rs:325
10672024-11-13T05:07:24.365ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641 file = illumos-utils/src/zone.rs:325
10682024-11-13T05:07:24.370ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb file = illumos-utils/src/zone.rs:325
10692024-11-13T05:07:24.494ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1 file = illumos-utils/src/zone.rs:363
10702024-11-13T05:07:24.500ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5 file = illumos-utils/src/zone.rs:363
10712024-11-13T05:07:24.504ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514 file = illumos-utils/src/zone.rs:363
10722024-11-13T05:07:24.509ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c file = illumos-utils/src/zone.rs:363
10732024-11-13T05:07:24.515ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a file = illumos-utils/src/zone.rs:363
10742024-11-13T05:07:24.520ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148 file = illumos-utils/src/zone.rs:363
10752024-11-13T05:07:24.525ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641 file = illumos-utils/src/zone.rs:363
10762024-11-13T05:07:24.530ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb file = illumos-utils/src/zone.rs:363
10772024-11-13T05:07:24.536ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208 file = illumos-utils/src/zone.rs:325
10782024-11-13T05:07:24.542ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317 file = illumos-utils/src/zone.rs:325
10792024-11-13T05:07:24.548ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871 file = illumos-utils/src/zone.rs:325
10802024-11-13T05:07:24.554ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde file = illumos-utils/src/zone.rs:325
10812024-11-13T05:07:24.560ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237 file = illumos-utils/src/zone.rs:325
10822024-11-13T05:07:24.567ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f file = illumos-utils/src/zone.rs:325
10832024-11-13T05:07:24.628ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
10842024-11-13T05:07:24.628ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208 file = illumos-utils/src/zone.rs:363
10852024-11-13T05:07:24.641ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317 file = illumos-utils/src/zone.rs:363
10862024-11-13T05:07:24.652ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871 file = illumos-utils/src/zone.rs:363
10872024-11-13T05:07:24.664ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde file = illumos-utils/src/zone.rs:363
10882024-11-13T05:07:24.707ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237 file = illumos-utils/src/zone.rs:363
10892024-11-13T05:07:24.719ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f file = illumos-utils/src/zone.rs:363
10902024-11-13T05:07:24.795ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
10912024-11-13T05:07:24.850ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
10922024-11-13T05:07:24.906ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
10932024-11-13T05:07:24.960ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
10942024-11-13T05:07:24.963ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
10952024-11-13T05:07:24.963ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
10962024-11-13T05:07:25.016ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte1", ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 207, 83]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
10972024-11-13T05:07:25.016ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = nexus
10982024-11-13T05:07:25.019ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
10992024-11-13T05:07:25.019ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
11002024-11-13T05:07:25.078ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte2", ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 205, 204]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
11012024-11-13T05:07:25.078ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = nexus
11022024-11-13T05:07:25.079ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
11032024-11-13T05:07:25.079ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
11042024-11-13T05:07:25.131ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte3", ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 148, 145]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
11052024-11-13T05:07:25.131ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = external_dns
11062024-11-13T05:07:25.132ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
11072024-11-13T05:07:25.133ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
11082024-11-13T05:07:25.184ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte4", ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 162, 5]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
11092024-11-13T05:07:25.184ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = nexus
11102024-11-13T05:07:25.185ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
11112024-11-13T05:07:25.185ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
11122024-11-13T05:07:25.236ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte5", ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 226, 83]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
11132024-11-13T05:07:25.236ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = external_dns
11142024-11-13T05:07:25.236ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:83
11152024-11-13T05:07:25.236ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:90
11162024-11-13T05:07:25.237ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:83
11172024-11-13T05:07:25.237ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:90
11182024-11-13T05:07:25.237ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:83
11192024-11-13T05:07:25.237ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:90
11202024-11-13T05:07:25.238ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:83
11212024-11-13T05:07:25.238ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:90
11222024-11-13T05:07:25.238ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:83
11232024-11-13T05:07:25.238ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:90
11242024-11-13T05:07:25.247ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:118
11252024-11-13T05:07:25.249ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
11262024-11-13T05:07:25.297ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:118
11272024-11-13T05:07:25.301ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:118
11282024-11-13T05:07:25.301ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
11292024-11-13T05:07:25.304ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:118
11302024-11-13T05:07:25.304ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
11312024-11-13T05:07:25.307ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:118
11322024-11-13T05:07:25.307ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
11332024-11-13T05:07:25.310ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
11342024-11-13T05:07:25.482ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea file = illumos-utils/src/zone.rs:325
11352024-11-13T05:07:25.489ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be file = illumos-utils/src/zone.rs:325
11362024-11-13T05:07:25.497ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66 file = illumos-utils/src/zone.rs:325
11372024-11-13T05:07:25.503ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5 file = illumos-utils/src/zone.rs:325
11382024-11-13T05:07:25.510ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf file = illumos-utils/src/zone.rs:325
11392024-11-13T05:07:25.516ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea file = illumos-utils/src/zone.rs:363
11402024-11-13T05:07:25.522ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be file = illumos-utils/src/zone.rs:363
11412024-11-13T05:07:25.528ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66 file = illumos-utils/src/zone.rs:363
11422024-11-13T05:07:25.535ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5 file = illumos-utils/src/zone.rs:363
11432024-11-13T05:07:25.542ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf file = illumos-utils/src/zone.rs:363
11442024-11-13T05:07:27.685ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService11'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::16'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_e770fce0-3dd5-4678-9037-68ea781dbb30/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::16'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='e8ac6632-89ff-44bb-aa07-61139dbe0513'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11452024-11-13T05:07:27.691ZINFOSledAgent (ServiceManager): Booting oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
11462024-11-13T05:07:27.742ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService16'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::f'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::f'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11472024-11-13T05:07:27.743ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
11482024-11-13T05:07:27.880ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService10'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::19'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_27d5a04d-ccb1-4219-851b-abc94499936f/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::19'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='c547f189-ad45-4dac-9003-f325989276ea'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11492024-11-13T05:07:27.880ZINFOSledAgent (ServiceManager): Booting oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
11502024-11-13T05:07:27.993ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService21'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::17'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_c2ce9d45-3d62-4234-a9db-c1d5c8884adc/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::17'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='0fa42ae1-90a2-471a-89a2-f643c10099f7'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11512024-11-13T05:07:27.993ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService20'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::15'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_8e821c87-2d05-4e93-b160-49a645a177c5/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::15'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='cd87613d-fb5c-442d-9b71-b36c00cd0863'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11522024-11-13T05:07:27.993ZINFOSledAgent (ServiceManager): Booting oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
11532024-11-13T05:07:28.004ZINFOSledAgent (ServiceManager): Booting oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
11542024-11-13T05:07:28.063ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService23'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_c7851ba0-6692-4564-8a56-a39ae4767989/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1a'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='47959000-1a44-4744-8b15-7405d1fb6e6e'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11552024-11-13T05:07:28.063ZINFOSledAgent (ServiceManager): Booting oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
11562024-11-13T05:07:28.098ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService19'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::13'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_c867bbf7-921c-4db1-887a-4002bcc6b446/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::13'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='7036e0ba-5466-4ee8-bf10-56f4be9e7c78'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11572024-11-13T05:07:28.099ZINFOSledAgent (ServiceManager): Booting oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
11582024-11-13T05:07:28.201ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 27.91367ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
11592024-11-13T05:07:28.214ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 41.103041ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
11602024-11-13T05:07:28.230ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 73.779941ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
11612024-11-13T05:07:28.243ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 120.12364ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
11622024-11-13T05:07:28.270ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 142.195526ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
11632024-11-13T05:07:28.288ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 47.24795ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
11642024-11-13T05:07:28.369ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 113.351063ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
11652024-11-13T05:07:28.535ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 292.071007ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
11662024-11-13T05:07:28.610ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 240.988268ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
11672024-11-13T05:07:28.657ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 79.145293ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
11682024-11-13T05:07:28.678ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 67.008068ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
11692024-11-13T05:07:36.893ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 59.798233ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
11702024-11-13T05:07:37.539ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 131.543938ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
11712024-11-13T05:07:37.958ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 485.267261ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
11722024-11-13T05:07:38.601ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 269.840245ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
11732024-11-13T05:07:38.601ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService18'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::12'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_8511034a-a894-457f-9cbc-98e6461f9f5a/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::12'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='6efd7366-fef9-42cc-88b2-055be55fdb34'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11742024-11-13T05:07:38.602ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService26'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::8'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte3'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::8]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.5:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11752024-11-13T05:07:39.027ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 67.91143ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
11762024-11-13T05:07:39.044ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 1.073130685s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:424 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
11772024-11-13T05:07:39.045ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 255.228591ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
11782024-11-13T05:07:39.278ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 54.353719ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
11792024-11-13T05:07:39.477ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService28'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::9'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte5'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::9]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.6:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11802024-11-13T05:07:39.489ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 137.571938ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
11812024-11-13T05:07:39.924ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 269.363103ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
11822024-11-13T05:07:39.938ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 869.98539ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
11832024-11-13T05:07:39.938ZINFOSledAgent (ServiceManager): Profile for oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService12'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::d'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/oximeter"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="id" value='671c6402-97d6-48bd-9991-5554bd96a514'/> <propval type="astring" name="address" value='[fd00:1122:3344:101::d]:12223'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11842024-11-13T05:07:39.938ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService15'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::10'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::10'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11852024-11-13T05:07:40.382ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 819.01394ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
11862024-11-13T05:07:40.383ZINFOSledAgent (ServiceManager): Booting oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
11872024-11-13T05:07:40.532ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
11882024-11-13T05:07:40.858ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 63.899527ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
11892024-11-13T05:07:40.870ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 375.80341ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
11902024-11-13T05:07:41.025ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 230.679667ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
11912024-11-13T05:07:41.025ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
11922024-11-13T05:07:41.041ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 222.881899ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
11932024-11-13T05:07:41.052ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 1.034308159s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
11942024-11-13T05:07:41.061ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 1.163011864s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
11952024-11-13T05:07:41.061ZINFOSledAgent (ServiceManager): Booting oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
11962024-11-13T05:07:41.073ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
11972024-11-13T05:07:41.092ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 155.247796ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
11982024-11-13T05:07:41.093ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService25'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte2'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11992024-11-13T05:07:41.093ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService27'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::c'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte4'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12002024-11-13T05:07:41.093ZINFOSledAgent (ServiceManager): Booting oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
12012024-11-13T05:07:41.100ZINFOSledAgent (ServiceManager): Booting oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
12022024-11-13T05:07:41.215ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 1.403935094s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
12032024-11-13T05:07:41.254ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 56.055534ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
12042024-11-13T05:07:41.265ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 213.944401ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
12052024-11-13T05:07:41.276ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 1.194335953s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
12062024-11-13T05:07:41.287ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 567.278091ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
12072024-11-13T05:07:41.297ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 419.739591ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
12082024-11-13T05:07:41.308ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 35.138654ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
12092024-11-13T05:07:41.318ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 39.930348ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
12102024-11-13T05:07:41.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 71.5495ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
12112024-11-13T05:07:41.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 59.199292ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
12122024-11-13T05:07:41.370ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 128.112193ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
12132024-11-13T05:07:41.390ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 52.435429ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
12142024-11-13T05:07:41.411ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 177.053819ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
12152024-11-13T05:07:41.426ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 266.499642ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
12162024-11-13T05:07:41.452ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 59.105243ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
12172024-11-13T05:07:41.492ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 847.182005ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
12182024-11-13T05:07:41.510ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 100.267712ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
12192024-11-13T05:07:41.522ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 213.195567ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
12202024-11-13T05:07:41.600ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 590.470614ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
12212024-11-13T05:07:41.623ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 234.685612ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
12222024-11-13T05:07:41.676ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 51.692693ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
12232024-11-13T05:07:41.707ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 585.454669ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
12242024-11-13T05:07:41.732ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 916.425142ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
12252024-11-13T05:07:41.742ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 130.252317ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
12262024-11-13T05:07:41.755ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 556.42796ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
12272024-11-13T05:07:41.869ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 805.236025ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
12282024-11-13T05:07:41.882ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 1.109603017s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
12292024-11-13T05:07:41.892ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 130.264831ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
12302024-11-13T05:07:42.034ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 331.17807ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
12312024-11-13T05:07:42.046ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 49.116782ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
12322024-11-13T05:07:42.100ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 1.45044977s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
12332024-11-13T05:07:42.111ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 78.314848ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
12342024-11-13T05:07:42.152ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 39.051379ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
12352024-11-13T05:07:42.202ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 185.871849ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
12362024-11-13T05:07:42.216ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 884.113398ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
12372024-11-13T05:07:42.225ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 73.013226ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
12382024-11-13T05:07:42.235ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 1.174259843s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
12392024-11-13T05:07:42.306ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 1.089067023s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
12402024-11-13T05:07:42.316ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 214.794064ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
12412024-11-13T05:07:42.328ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 1.149075466s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
12422024-11-13T05:07:42.350ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 1.179424328s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
12432024-11-13T05:07:42.374ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 652.838131ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
12442024-11-13T05:07:42.397ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 347.296763ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
12452024-11-13T05:07:42.481ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 1.432229815s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
12462024-11-13T05:07:42.541ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 285.861669ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
12472024-11-13T05:07:42.630ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 708.27912ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
12482024-11-13T05:07:42.657ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 837.796749ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
12492024-11-13T05:07:42.682ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 1.468310889s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
12502024-11-13T05:07:42.753ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 842.795927ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
12512024-11-13T05:07:42.838ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 458.363615ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
12522024-11-13T05:07:43.003ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 1.394057061s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
12532024-11-13T05:07:43.009ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService13'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::e'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::e'/> <propval type="astring" name="listen_port" value='8123'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::e]:8888'/> <propval type="astring" name="ch_address" value='[fd00:1122:3344:101::e]:8123'/> <propval type="astring" name="ch_binary" value='/opt/oxide/clickhouse/clickhouse'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12542024-11-13T05:07:43.010ZINFOSledAgent (ServiceManager): Booting oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
12552024-11-13T05:07:43.036ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 503.280183ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
12562024-11-13T05:07:43.109ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 1.274677523s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
12572024-11-13T05:07:43.306ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 873.470852ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
12582024-11-13T05:07:43.347ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 1.484058735s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
12592024-11-13T05:07:43.404ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 548.3392ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
12602024-11-13T05:07:43.419ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 602.782325ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
12612024-11-13T05:07:43.425ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 41.706171ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
12622024-11-13T05:07:43.476ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 90.626626ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
12632024-11-13T05:07:43.486ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 1.062759298s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
12642024-11-13T05:07:43.503ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 829.162199ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
12652024-11-13T05:07:43.538ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 1.426711687s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
12662024-11-13T05:07:43.548ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 704.776973ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
12672024-11-13T05:07:43.560ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 561.362998ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
12682024-11-13T05:07:43.574ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 168.074489ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
12692024-11-13T05:07:43.604ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 634.619506ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
12702024-11-13T05:07:43.751ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 251.011563ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
12712024-11-13T05:07:43.925ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 861.409378ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
12722024-11-13T05:07:43.962ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 1.372137686s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
12732024-11-13T05:07:44.011ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 414.513931ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
12742024-11-13T05:07:44.030ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 704.685017ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
12752024-11-13T05:07:44.130ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 1.190331038s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
12762024-11-13T05:07:44.158ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 1.313586806s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
12772024-11-13T05:07:44.187ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 640.469857ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
12782024-11-13T05:07:44.248ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 661.942624ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
12792024-11-13T05:07:44.261ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 623.565278ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
12802024-11-13T05:07:44.341ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 1.39990408s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
12812024-11-13T05:07:44.393ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 745.448563ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
12822024-11-13T05:07:44.405ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 1.400277229s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
12832024-11-13T05:07:44.434ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 1.311002481s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
12842024-11-13T05:07:44.558ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 1.401201051s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
12852024-11-13T05:07:44.634ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService17'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::18'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_c9fdd7dd-67c5-4597-a857-03bc9769dc87/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::18'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='b16b5b79-1c9e-4196-ac34-0ee84795127c'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
12862024-11-13T05:07:44.634ZINFOSledAgent (ServiceManager): Booting oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
12872024-11-13T05:07:44.744ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 602.735569ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
12882024-11-13T05:07:44.797ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 1.079067717s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
12892024-11-13T05:07:44.827ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 28.890381ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
12902024-11-13T05:07:44.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 1.248555097s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
12912024-11-13T05:07:44.845ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 755.30914ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
12922024-11-13T05:07:44.864ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 65.718662ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
12932024-11-13T05:07:44.894ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 1.062837795s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
12942024-11-13T05:07:44.919ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 654.270949ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
12952024-11-13T05:07:44.938ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 219.767073ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
12962024-11-13T05:07:44.980ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 964.486284ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
12972024-11-13T05:07:45.261ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 981.35544ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
12982024-11-13T05:07:45.288ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 520.439771ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
12992024-11-13T05:07:45.400ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 598.475168ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
13002024-11-13T05:07:45.528ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 1.349769158s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
13012024-11-13T05:07:45.549ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 918.376767ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
13022024-11-13T05:07:45.572ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 1.043239898s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
13032024-11-13T05:07:45.598ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 871.297652ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
13042024-11-13T05:07:45.622ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 1.149038666s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
13052024-11-13T05:07:45.755ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 803.296298ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
13062024-11-13T05:07:45.768ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 1.210495878s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
13072024-11-13T05:07:45.820ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 532.766962ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
13082024-11-13T05:07:45.831ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 1.179811062s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
13092024-11-13T05:07:45.887ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 770.139124ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
13102024-11-13T05:07:45.958ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 596.793849ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
13112024-11-13T05:07:45.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 847.474046ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
13122024-11-13T05:07:45.981ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 834.487ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
13132024-11-13T05:07:46.009ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 1.180036263s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
13142024-11-13T05:07:46.100ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 816.202451ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
13152024-11-13T05:07:46.254ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 518.502883ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
13162024-11-13T05:07:46.366ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 1.051658863s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
13172024-11-13T05:07:46.480ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 902.284003ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
13182024-11-13T05:07:46.491ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 1.359766626s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
13192024-11-13T05:07:46.569ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 1.133430418s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
13202024-11-13T05:07:46.581ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 810.196389ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
13212024-11-13T05:07:46.629ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 576.040937ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
13222024-11-13T05:07:46.671ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 1.030898046s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
13232024-11-13T05:07:46.783ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 927.032854ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
13242024-11-13T05:07:46.794ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 823.365854ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
13252024-11-13T05:07:46.827ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 1.178418056s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
13262024-11-13T05:07:46.838ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 1.44685905s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
13272024-11-13T05:07:46.890ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 520.675402ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
13282024-11-13T05:07:46.927ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 1.126639016s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
13292024-11-13T05:07:46.990ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 879.910899ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
13302024-11-13T05:07:47.022ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 1.131606971s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
13312024-11-13T05:07:47.201ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 1.321562942s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
13322024-11-13T05:07:47.217ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 578.230908ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
13332024-11-13T05:07:47.395ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 933.477718ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
13342024-11-13T05:07:47.404ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 916.609563ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
13352024-11-13T05:07:47.421ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 679.528807ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
13362024-11-13T05:07:47.430ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 831.383156ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
13372024-11-13T05:07:47.592ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService24'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte1'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13382024-11-13T05:07:47.592ZINFOSledAgent (ServiceManager): Booting oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
13392024-11-13T05:07:47.629ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 1.057793809s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
13402024-11-13T05:07:47.712ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 1.212132879s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
13412024-11-13T05:07:47.776ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 1.131808315s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
13422024-11-13T05:07:47.787ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 1.138147346s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
13432024-11-13T05:07:47.809ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 985.610036ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
13442024-11-13T05:07:47.862ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 1.195811473s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
13452024-11-13T05:07:47.873ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea") failed: Property not found. retry in 73.919937ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
13462024-11-13T05:07:47.883ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 715.368802ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
13472024-11-13T05:07:47.958ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea") failed: Property not found. retry in 56.240144ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
13482024-11-13T05:07:48.019ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 1.460839921s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
13492024-11-13T05:07:48.030ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea") failed: Property not found. retry in 114.380274ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
13502024-11-13T05:07:48.065ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 1.442764995s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
13512024-11-13T05:07:48.110ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 1.354724305s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
13522024-11-13T05:07:48.153ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea") failed: Property not found. retry in 329.56422ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
13532024-11-13T05:07:48.164ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 733.444505ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
13542024-11-13T05:07:48.271ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 1.049302103s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
13552024-11-13T05:07:48.294ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 1.147409864s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
13562024-11-13T05:07:48.331ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 1.339695439s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
13572024-11-13T05:07:48.339ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 507.042461ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
13582024-11-13T05:07:48.494ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea") failed: Property not found. retry in 701.992162ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
13592024-11-13T05:07:48.531ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 998.899749ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
13602024-11-13T05:07:48.607ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 790.149154ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
13612024-11-13T05:07:48.696ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 722.968556ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
13622024-11-13T05:07:48.804ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 966.687236ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
13632024-11-13T05:07:48.854ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 938.996885ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
13642024-11-13T05:07:48.905ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 629.905106ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
13652024-11-13T05:07:48.915ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 841.038101ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
13662024-11-13T05:07:48.933ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 743.456318ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
13672024-11-13T05:07:48.941ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 890.211225ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
13682024-11-13T05:07:49.068ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 940.464351ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
13692024-11-13T05:07:49.206ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea") failed: Property not found. retry in 1.155908062s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
13702024-11-13T05:07:49.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 707.216493ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
13712024-11-13T05:07:49.406ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 1.335618411s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
13722024-11-13T05:07:49.427ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 1.135908813s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
13732024-11-13T05:07:49.450ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 763.48839ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
13742024-11-13T05:07:49.474ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 568.064014ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
13752024-11-13T05:07:49.489ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 1.210554675s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
13762024-11-13T05:07:49.587ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 1.337219251s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
13772024-11-13T05:07:49.630ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 1.431289435s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
13782024-11-13T05:07:49.638ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 922.766616ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
13792024-11-13T05:07:49.701ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 1.025543726s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
13802024-11-13T05:07:49.711ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 614.664745ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
13812024-11-13T05:07:49.766ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 988.97901ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
13822024-11-13T05:07:49.778ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 598.718809ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
13832024-11-13T05:07:49.803ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 971.079849ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
13842024-11-13T05:07:49.839ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 784.950661ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
13852024-11-13T05:07:50.018ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 1.079578879s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
13862024-11-13T05:07:50.046ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 770.817278ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
13872024-11-13T05:07:50.054ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 880.070793ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
13882024-11-13T05:07:50.221ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 1.120532187s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
13892024-11-13T05:07:50.336ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 1.038941735s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
13902024-11-13T05:07:50.371ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea") failed: Property not found. retry in 1.494881502s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
13912024-11-13T05:07:50.386ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 1.091502666s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
13922024-11-13T05:07:50.571ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 1.274159944s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
13932024-11-13T05:07:50.580ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 910.280292ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
13942024-11-13T05:07:50.634ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 1.472827478s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
13952024-11-13T05:07:50.709ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 1.3068871s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
13962024-11-13T05:07:50.735ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 615.999865ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
13972024-11-13T05:07:50.751ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 660.368663ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
13982024-11-13T05:07:50.764ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 893.424923ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
13992024-11-13T05:07:50.783ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 964.394965ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
14002024-11-13T05:07:50.826ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 1.071698755s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
14012024-11-13T05:07:50.933ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 1.282594536s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
14022024-11-13T05:07:50.944ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 903.529532ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
14032024-11-13T05:07:51.072ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 825.444254ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
14042024-11-13T05:07:51.107ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 587.242275ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
14052024-11-13T05:07:51.352ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 1.343395068s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
14062024-11-13T05:07:51.360ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 1.360647615s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
14072024-11-13T05:07:51.383ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 1.485719707s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
14082024-11-13T05:07:51.420ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 1.393815092s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
14092024-11-13T05:07:51.487ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 841.447747ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
14102024-11-13T05:07:51.499ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 1.357378873s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
14112024-11-13T05:07:51.666ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 610.565749ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
14122024-11-13T05:07:51.703ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 1.118768232s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
14132024-11-13T05:07:51.756ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 1.461026691s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
14142024-11-13T05:07:51.854ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 984.263417ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
14152024-11-13T05:07:51.861ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 1.286727114s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
14162024-11-13T05:07:51.875ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea") failed: Property not found. retry in 1.306951717s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
14172024-11-13T05:07:51.906ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 718.057048ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
14182024-11-13T05:07:51.914ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 1.389096243s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
14192024-11-13T05:07:52.026ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 1.18738315s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
14202024-11-13T05:07:52.115ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 1.351114748s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
14212024-11-13T05:07:52.225ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 1.093668112s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
14222024-11-13T05:07:52.286ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 621.347148ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
14232024-11-13T05:07:52.338ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 1.280385839s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
14242024-11-13T05:07:52.633ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 728.956937ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
14252024-11-13T05:07:52.705ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 1.084672376s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
14262024-11-13T05:07:52.730ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 1.333275184s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
14272024-11-13T05:07:52.823ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 1.07998368s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
14282024-11-13T05:07:52.830ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 625.790078ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
14292024-11-13T05:07:52.847ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 936.792303ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
14302024-11-13T05:07:52.865ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 1.475879723s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
14312024-11-13T05:07:52.878ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 644.815108ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
14322024-11-13T05:07:52.915ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 1.226408162s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
14332024-11-13T05:07:53.157ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 670.241099ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
14342024-11-13T05:07:53.191ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea") failed: Property not found. retry in 1.434380854s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
14352024-11-13T05:07:53.223ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 1.142435444s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
14362024-11-13T05:07:53.231ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 1.192440675s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
14372024-11-13T05:07:53.312ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 1.224163333s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
14382024-11-13T05:07:53.327ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 1.446805412s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
14392024-11-13T05:07:53.371ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 1.344576142s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
14402024-11-13T05:07:53.465ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 1.394927106s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
14412024-11-13T05:07:53.474ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 1.494392096s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
14422024-11-13T05:07:53.530ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 1.323183253s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
14432024-11-13T05:07:53.628ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 800.076185ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
14442024-11-13T05:07:53.691ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService22'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::11'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::11'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14452024-11-13T05:07:53.735ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237
14462024-11-13T05:07:53.809ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 575.747974ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
14472024-11-13T05:07:53.821ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 925.515268ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
14482024-11-13T05:07:53.826ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService14'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::14'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_204f40c4-4e09-4726-a368-a3da2ac1b831/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::14'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='99b3262c-5a50-4d85-b344-62be9401b7ac'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14492024-11-13T05:07:53.990ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 1.340451593s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
14502024-11-13T05:07:53.990ZINFOSledAgent (ServiceManager): Booting oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a
14512024-11-13T05:07:54.009ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:37969
14522024-11-13T05:07:54.016ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 607.297067ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
14532024-11-13T05:07:54.079ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1247 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:37969 req_id = 8e63bbaf-975b-4801-b3f8-c45d72d0cd28 response_code = 200 uri = /baseboard
14542024-11-13T05:07:54.083ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 1.274645927s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
14552024-11-13T05:07:54.153ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 1.395905165s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
14562024-11-13T05:07:54.331ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237") failed: Property not found. retry in 59.784645ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237
14572024-11-13T05:07:54.351ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 649.621073ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
14582024-11-13T05:07:54.375ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 560.8768ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
14592024-11-13T05:07:54.394ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 934.04661ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
14602024-11-13T05:07:54.402ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237") failed: Property not found. retry in 140.439635ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237
14612024-11-13T05:07:54.432ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 869.02856ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
14622024-11-13T05:07:54.440ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 706.413773ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
14632024-11-13T05:07:54.514ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a") failed: Property not found. retry in 34.288545ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a
14642024-11-13T05:07:54.546ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 626.242099ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
14652024-11-13T05:07:54.552ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237") failed: Property not found. retry in 211.640815ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237
14662024-11-13T05:07:54.560ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a") failed: Property not found. retry in 72.302552ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a
14672024-11-13T05:07:54.631ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 1.464350721s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
14682024-11-13T05:07:54.639ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea") failed: Property not found. retry in 1.040093373s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
14692024-11-13T05:07:54.646ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a") failed: Property not found. retry in 266.776781ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a
14702024-11-13T05:07:54.725ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 886.445983ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
14712024-11-13T05:07:54.757ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 1.23692203s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
14722024-11-13T05:07:54.774ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237") failed: Property not found. retry in 498.589077ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237
14732024-11-13T05:07:54.783ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 1.106036518s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
14742024-11-13T05:07:54.866ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 1.209320678s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
14752024-11-13T05:07:54.875ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 884.281931ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
14762024-11-13T05:07:54.928ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a") failed: Property not found. retry in 216.073173ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a
14772024-11-13T05:07:54.948ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 1.030207453s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
14782024-11-13T05:07:54.980ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 718.262049ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
14792024-11-13T05:07:55.011ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 1.42454127s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
14802024-11-13T05:07:55.156ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a") failed: Property not found. retry in 845.750205ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a
14812024-11-13T05:07:55.166ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 1.062392652s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
14822024-11-13T05:07:55.184ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 566.721957ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
14832024-11-13T05:07:55.287ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237") failed: Property not found. retry in 1.160239871s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237
14842024-11-13T05:07:55.312ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 1.231074778s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
14852024-11-13T05:07:55.340ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 717.755245ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
14862024-11-13T05:07:55.350ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 1.436822816s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
14872024-11-13T05:07:55.369ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 1.073713269s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
14882024-11-13T05:07:55.560ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 714.131584ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
14892024-11-13T05:07:55.624ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 540.383222ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
14902024-11-13T05:07:55.693ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea") failed: Property not found. retry in 1.418171203s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
14912024-11-13T05:07:55.712ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 582.515237ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
14922024-11-13T05:07:55.769ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 1.326158215s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
14932024-11-13T05:07:55.787ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 1.232967277s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
14942024-11-13T05:07:55.903ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 667.323ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
14952024-11-13T05:07:55.990ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 886.493563ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
14962024-11-13T05:07:56.007ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 771.8808ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
14972024-11-13T05:07:56.017ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a") failed: Property not found. retry in 796.314316ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a
14982024-11-13T05:07:56.069ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 980.923582ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
14992024-11-13T05:07:56.086ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 1.034596282s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
15002024-11-13T05:07:56.094ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 1.426916961s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:424 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
15012024-11-13T05:07:56.109ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 849.180035ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
15022024-11-13T05:07:56.175ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 933.503295ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
15032024-11-13T05:07:56.242ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 536.975169ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
15042024-11-13T05:07:56.285ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 1.034042099s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
15052024-11-13T05:07:56.306ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 910.320668ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
15062024-11-13T05:07:56.448ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 1.247919563s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
15072024-11-13T05:07:56.459ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237") failed: Property not found. retry in 585.205571ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237
15082024-11-13T05:07:56.469ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 1.441409232s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
15092024-11-13T05:07:56.558ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 593.198551ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
15102024-11-13T05:07:56.585ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 807.119917ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
15112024-11-13T05:07:56.793ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 508.323668ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
15122024-11-13T05:07:56.805ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 1.439725533s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
15132024-11-13T05:07:56.817ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 971.992608ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
15142024-11-13T05:07:56.830ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a") failed: Property not found. retry in 1.392843747s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a
15152024-11-13T05:07:56.889ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 735.996562ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
15162024-11-13T05:07:56.973ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 1.103068423s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
15172024-11-13T05:07:57.035ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 1.041254157s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
15182024-11-13T05:07:57.061ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237") failed: Property not found. retry in 1.116681098s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237
15192024-11-13T05:07:57.081ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 946.803533ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
15202024-11-13T05:07:57.112ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 1.132229362s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
15212024-11-13T05:07:57.131ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 1.070607305s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
15222024-11-13T05:07:57.149ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea") failed: Property not found. retry in 1.130158087s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
15232024-11-13T05:07:57.170ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 887.982454ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
15242024-11-13T05:07:57.188ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 907.923015ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
15252024-11-13T05:07:57.234ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 624.170926ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
15262024-11-13T05:07:57.318ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 1.203464625s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
15272024-11-13T05:07:57.335ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 1.475012017s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
15282024-11-13T05:07:57.420ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 1.452944435s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
15292024-11-13T05:07:57.643ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 951.728509ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
15302024-11-13T05:07:57.715ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 1.367098995s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
15312024-11-13T05:07:57.809ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 797.682957ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
15322024-11-13T05:07:57.877ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 901.236069ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
15332024-11-13T05:07:57.935ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 999.171213ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
15342024-11-13T05:07:58.047ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 552.038605ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
15352024-11-13T05:07:58.080ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 1.197327892s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
15362024-11-13T05:07:58.097ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 1.082057622s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
15372024-11-13T05:07:58.119ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 821.254559ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
15382024-11-13T05:07:58.136ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 1.384452137s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
15392024-11-13T05:07:58.194ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237") failed: Property not found. retry in 951.752717ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237
15402024-11-13T05:07:58.221ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 1.36297612s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
15412024-11-13T05:07:58.239ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a") failed: Property not found. retry in 1.259245624s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a
15422024-11-13T05:07:58.264ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 546.264051ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
15432024-11-13T05:07:58.280ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 1.119021875s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
15442024-11-13T05:07:58.300ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea") failed: Property not found. retry in 1.301453621s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
15452024-11-13T05:07:58.534ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 1.368500776s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
15462024-11-13T05:07:58.605ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 661.625348ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
15472024-11-13T05:07:58.614ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 928.016554ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
15482024-11-13T05:07:58.622ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 567.959663ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
15492024-11-13T05:07:58.794ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 1.000412631s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
15502024-11-13T05:07:58.821ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 536.191033ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
15512024-11-13T05:07:58.832ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 1.131448733s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
15522024-11-13T05:07:58.885ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 882.469623ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
15532024-11-13T05:07:58.944ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 973.262021ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
15542024-11-13T05:07:58.954ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 824.382337ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
15552024-11-13T05:07:59.092ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 838.822175ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
15562024-11-13T05:07:59.157ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237") failed: Property not found. retry in 653.63904ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237
15572024-11-13T05:07:59.188ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 1.060853049s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
15582024-11-13T05:07:59.198ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 1.242543939s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
15592024-11-13T05:07:59.275ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 1.441298252s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
15602024-11-13T05:07:59.286ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 511.08699ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
15612024-11-13T05:07:59.368ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 1.083323305s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
15622024-11-13T05:07:59.410ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 1.404854152s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
15632024-11-13T05:07:59.509ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a") failed: Property not found. retry in 1.499128692s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a
15642024-11-13T05:07:59.530ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 1.499716262s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
15652024-11-13T05:07:59.554ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 695.85829ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
15662024-11-13T05:07:59.595ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 1.349081582s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
15672024-11-13T05:07:59.613ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea") failed: Property not found. retry in 532.540812ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
15682024-11-13T05:07:59.780ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 549.741602ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
15692024-11-13T05:07:59.792ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 878.288383ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
15702024-11-13T05:07:59.805ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 1.072579923s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
15712024-11-13T05:07:59.816ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 1.183322435s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
15722024-11-13T05:07:59.826ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237") failed: Property not found. retry in 759.246334ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237
15732024-11-13T05:07:59.916ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 747.262186ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
15742024-11-13T05:07:59.929ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 767.980387ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
15752024-11-13T05:07:59.947ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 1.274765945s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
15762024-11-13T05:07:59.979ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 1.160358014s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
15772024-11-13T05:08:00.158ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea") failed: Property not found. retry in 1.427996429s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
15782024-11-13T05:08:00.261ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 1.235693577s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
15792024-11-13T05:08:00.273ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 507.744664ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
15802024-11-13T05:08:00.341ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 656.845258ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
15812024-11-13T05:08:00.459ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 933.78974ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
15822024-11-13T05:08:00.471ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 552.099729ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
15832024-11-13T05:08:00.606ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237") failed: Property not found. retry in 1.38241567s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237
15842024-11-13T05:08:00.679ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 610.609284ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
15852024-11-13T05:08:00.692ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 624.014385ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
15862024-11-13T05:08:00.711ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 792.979965ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
15872024-11-13T05:08:00.732ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 567.930602ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
15882024-11-13T05:08:00.796ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb") failed: Property not found. retry in 1.256924588s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
15892024-11-13T05:08:00.829ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 1.187996526s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
15902024-11-13T05:08:00.897ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 988.957146ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
15912024-11-13T05:08:00.956ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 1.325245349s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
15922024-11-13T05:08:01.013ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514") failed: Property not found. retry in 1.304105399s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
15932024-11-13T05:08:01.027ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a") failed: Property not found. retry in 1.006933453s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a
15942024-11-13T05:08:01.044ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 700.184759ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
15952024-11-13T05:08:01.060ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 1.470200538s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
15962024-11-13T05:08:01.073ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 765.385683ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
15972024-11-13T05:08:01.157ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871") failed: Property not found. retry in 1.385817321s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
15982024-11-13T05:08:01.237ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 576.62246ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
15992024-11-13T05:08:01.308ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208") failed: Property not found. retry in 1.203089213s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
16002024-11-13T05:08:01.321ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 830.597981ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
16012024-11-13T05:08:01.339ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 984.505796ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
16022024-11-13T05:08:01.410ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5") failed: Property not found. retry in 1.416615079s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
16032024-11-13T05:08:01.511ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 528.726582ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
16042024-11-13T05:08:01.525ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 1.412045012s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
16052024-11-13T05:08:01.602ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea") failed: Property not found. retry in 559.53096ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
16062024-11-13T05:08:01.757ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317") failed: Property not found. retry in 1.223419681s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
16072024-11-13T05:08:01.833ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66") failed: Property not found. retry in 1.226269023s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
16082024-11-13T05:08:01.856ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5") failed: Property not found. retry in 1.299954189s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
16092024-11-13T05:08:01.907ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641") failed: Property not found. retry in 820.181343ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
16102024-11-13T05:08:02.006ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237") failed: Property not found. retry in 1.262253284s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237
16112024-11-13T05:08:02.036ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde") failed: Property not found. retry in 1.159777119s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
16122024-11-13T05:08:02.053ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a") failed: Property not found. retry in 1.039767264s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a
16132024-11-13T05:08:02.072ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c") failed: Property not found. retry in 1.350889067s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
16142024-11-13T05:08:02.174ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 1.083032361s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
16152024-11-13T05:08:02.292ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1") failed: Property not found. retry in 771.484328ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
16162024-11-13T05:08:02.357ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148") failed: Property not found. retry in 630.093444ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
16172024-11-13T05:08:02.564ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be") failed: Property not found. retry in 723.025369ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
16182024-11-13T05:08:02.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f") failed: Property not found. retry in 850.958696ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
16192024-11-13T05:08:03.147ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a") failed: Property not found. retry in 1.199589927s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a
16202024-11-13T05:08:03.285ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf") failed: Property not found. retry in 1.477537637s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
16212024-11-13T05:08:03.309ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237") failed: Property not found. retry in 890.350382ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237
16222024-11-13T05:08:04.366ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a") failed: Property not found. retry in 999.903779ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a
16232024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_f8c37587-dd71-4b4c-84d0-02d3d83b3ff1
16242024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_d3ea4bae-2501-4144-a858-606f20fb94c5
16252024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_oximeter_671c6402-97d6-48bd-9991-5554bd96a514
16262024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_clickhouse_d21b9c90-434a-419c-aa82-907917c5d09c
16272024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_ed7f3aff-8a79-4a12-a77f-6a877e7ba03a
16282024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_pantry_98c5824c-c3f0-4916-bdb4-2d59cf1e6148
16292024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_pantry_dfbd3921-57ae-428e-b0a3-4f8aad3d8641
16302024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_51d35f2d-6f54-4059-b52e-080f645ee9eb
16312024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_nexus_e181617d-520a-429d-85ef-385f6dde38ea
16322024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_nexus_7641195e-73b4-4972-ad51-4fcacb36a4be
16332024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_4134cecf-46e8-4040-95d2-8570b8a2f208
16342024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_external_dns_eb4a98a4-3147-445f-9881-33f4dffeca66
16352024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_nexus_6c4893ca-1b18-4910-8b18-504e5fcb8dc5
16362024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_02279d36-b72f-47ae-859e-12e5a6192317
16372024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_a9fdd6bc-028a-4967-ae6c-005966661871
16382024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_c131db8a-f8d3-4b52-9eef-d815a35dcdde
16392024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_external_dns_038e4c2b-2408-42f1-b025-c88f1167faaf
16402024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_pantry_42070343-2876-47fd-85a9-8ad8c72fa237
16412024-11-13T05:08:05.402ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_cb2d36d7-52c5-4aa0-bc43-f9e463290a5f
16422024-11-13T05:08:05.403ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
16432024-11-13T05:08:05.406ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
16442024-11-13T05:08:05.408ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 42420181 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:51011 req_id = d1d7ea3a-9fe2-41ca-bbfa-373ace712476 response_code = 204 uri = /omicron-zones
16452024-11-13T05:08:05.414ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1446
16462024-11-13T05:08:05.414ZINFOSledAgent (RSS): Writing ledger to /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
16472024-11-13T05:08:05.414ZINFOSledAgent (RSS): Writing ledger to /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
16482024-11-13T05:08:05.419ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:787
16492024-11-13T05:08:05.420ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::c]:12221 file = sled-agent/src/rack_setup/service.rs:799
16502024-11-13T05:08:05.453ZINFOSledAgent (RSS): rack_network_config: RackNetworkConfigV2 { bfd: [], bgp: [], infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], autoneg: false, bgp_peers: [], lldp: None, port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, rib_priority: None, vlan_id: None, }, ], switch: Switch0, tx_eq: None, uplink_port_fec: None, uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, } file = sled-agent/src/rack_setup/service.rs:1011
16512024-11-13T05:08:05.766ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/b7b41d81-f310-4949-894b-9f77b62a6c9b/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1073
16522024-11-13T05:08:05.768ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 3.143332279s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:424 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
16532024-11-13T05:08:05.831ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/b7b41d81-f310-4949-894b-9f77b62a6c9b/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1073
16542024-11-13T05:08:05.896ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/b7b41d81-f310-4949-894b-9f77b62a6c9b/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1073
16552024-11-13T05:08:06.011ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/b7b41d81-f310-4949-894b-9f77b62a6c9b/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1073
16562024-11-13T05:08:06.151ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/b7b41d81-f310-4949-894b-9f77b62a6c9b/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1073
16572024-11-13T05:08:06.393ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/b7b41d81-f310-4949-894b-9f77b62a6c9b/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1073
16582024-11-13T05:08:06.753ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/b7b41d81-f310-4949-894b-9f77b62a6c9b/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1073
16592024-11-13T05:08:06.956ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/b7b41d81-f310-4949-894b-9f77b62a6c9b/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1073
16602024-11-13T05:08:07.335ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/b7b41d81-f310-4949-894b-9f77b62a6c9b/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1073
16612024-11-13T05:08:07.567ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/b7b41d81-f310-4949-894b-9f77b62a6c9b/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1073
16622024-11-13T05:08:08.019ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/b7b41d81-f310-4949-894b-9f77b62a6c9b/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1073
16632024-11-13T05:08:08.518ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/racks/b7b41d81-f310-4949-894b-9f77b62a6c9b/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1073
16642024-11-13T05:08:09.033ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 7.264025661s error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"7cac3555-c8b0-4fc0-9324-c0611cd80179\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Wed, 13 Nov 2024 05:08:09 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"7cac3555-c8b0-4fc0-9324-c0611cd80179\\" }" file = oximeter/producer/src/lib.rs:424 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
16652024-11-13T05:08:09.526ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "4fbeff8b-c567-4215-804d-ead34c8e6a17", "content-length": "173", "date": "Wed, 13 Nov 2024 05:08:09 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"b7b41d81-f310-4949-894b-9f77b62a6c9b\"", request_id: "4fbeff8b-c567-4215-804d-ead34c8e6a17" } file = sled-agent/src/rack_setup/service.rs:1073
16662024-11-13T05:08:10.275ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "211cfc22-9e7b-4ffa-9eac-c47f44d3cd5b", "content-length": "173", "date": "Wed, 13 Nov 2024 05:08:10 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"b7b41d81-f310-4949-894b-9f77b62a6c9b\"", request_id: "211cfc22-9e7b-4ffa-9eac-c47f44d3cd5b" } file = sled-agent/src/rack_setup/service.rs:1073
16672024-11-13T05:08:12.332ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "2bef84c8-4604-4125-bde6-3d0c1194a8f9", "content-length": "173", "date": "Wed, 13 Nov 2024 05:08:11 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"4d997277-f640-48d7-af56-6df261d5afe1\"", request_id: "2bef84c8-4604-4125-bde6-3d0c1194a8f9" } file = sled-agent/src/rack_setup/service.rs:1073
16682024-11-13T05:08:14.043ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "a493e4f4-7e87-4cd0-a664-2b6f227ea308", "content-length": "173", "date": "Wed, 13 Nov 2024 05:08:14 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"4d997277-f640-48d7-af56-6df261d5afe1\"", request_id: "a493e4f4-7e87-4cd0-a664-2b6f227ea308" } file = sled-agent/src/rack_setup/service.rs:1073
16692024-11-13T05:08:15.871ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "873a2080-a3fe-4a8e-9fc5-b1d6160e5781", "content-length": "173", "date": "Wed, 13 Nov 2024 05:08:15 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"4d997277-f640-48d7-af56-6df261d5afe1\"", request_id: "873a2080-a3fe-4a8e-9fc5-b1d6160e5781" } file = sled-agent/src/rack_setup/service.rs:1073
16702024-11-13T05:08:17.370ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "aef91389-36e1-4416-bbdc-259c761d703f", "content-length": "173", "date": "Wed, 13 Nov 2024 05:08:16 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"4d997277-f640-48d7-af56-6df261d5afe1\"", request_id: "aef91389-36e1-4416-bbdc-259c761d703f" } file = sled-agent/src/rack_setup/service.rs:1073
16712024-11-13T05:08:19.521ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "6318f92f-35e9-408f-aca9-038f332fe31a", "content-length": "173", "date": "Wed, 13 Nov 2024 05:08:19 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"4d997277-f640-48d7-af56-6df261d5afe1\"", request_id: "6318f92f-35e9-408f-aca9-038f332fe31a" } file = sled-agent/src/rack_setup/service.rs:1073
16722024-11-13T05:08:22.797ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "859e08c7-5ecc-4263-9d16-b9a71b4a30a6", "content-length": "173", "date": "Wed, 13 Nov 2024 05:08:22 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"4d997277-f640-48d7-af56-6df261d5afe1\"", request_id: "859e08c7-5ecc-4263-9d16-b9a71b4a30a6" } file = sled-agent/src/rack_setup/service.rs:1073
16732024-11-13T05:08:24.117ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:63053
16742024-11-13T05:08:24.117ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 38 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:63053 req_id = 9f610641-9f38-49ef-9c54-5b6ddc1e5809 response_code = 200 uri = /baseboard
16752024-11-13T05:08:27.575ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "9331dcad-ce50-46bc-b381-d3dfe1afbb70", "content-length": "173", "date": "Wed, 13 Nov 2024 05:08:27 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"4d997277-f640-48d7-af56-6df261d5afe1\"", request_id: "9331dcad-ce50-46bc-b381-d3dfe1afbb70" } file = sled-agent/src/rack_setup/service.rs:1073
16762024-11-13T05:08:30.788ZINFOSledAgent (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:38057 remote_addr = [fd00:1122:3344:101::d]:39949 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
16772024-11-13T05:08:30.810ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 21584 local_addr = [fd00:1122:3344:101::1]:38057 method = GET remote_addr = [fd00:1122:3344:101::d]:39949 req_id = 41391943-4137-446d-881b-b22052da9c0a response_code = 200 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 uri = /4d997277-f640-48d7-af56-6df261d5afe1
16782024-11-13T05:08:30.946ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "7c2c7473-7257-4f19-adb2-467a917b00d0", "content-length": "173", "date": "Wed, 13 Nov 2024 05:08:30 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"4d997277-f640-48d7-af56-6df261d5afe1\"", request_id: "7c2c7473-7257-4f19-adb2-467a917b00d0" } file = sled-agent/src/rack_setup/service.rs:1073
16792024-11-13T05:08:35.423ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "27ee9759-5b87-4fab-8ac2-5a1dcca0115e", "content-length": "173", "date": "Wed, 13 Nov 2024 05:08:34 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"4d997277-f640-48d7-af56-6df261d5afe1\"", request_id: "27ee9759-5b87-4fab-8ac2-5a1dcca0115e" } file = sled-agent/src/rack_setup/service.rs:1073
16802024-11-13T05:08:41.266ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "d88590c0-bf32-4be0-b551-9916e4d9758f", "content-length": "173", "date": "Wed, 13 Nov 2024 05:08:41 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"4d997277-f640-48d7-af56-6df261d5afe1\"", request_id: "d88590c0-bf32-4be0-b551-9916e4d9758f" } file = sled-agent/src/rack_setup/service.rs:1073
16812024-11-13T05:08:47.518ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "aa88cbae-b856-44a1-99f9-50cc095d447a", "content-length": "173", "date": "Wed, 13 Nov 2024 05:08:47 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"4d997277-f640-48d7-af56-6df261d5afe1\"", request_id: "aa88cbae-b856-44a1-99f9-50cc095d447a" } file = sled-agent/src/rack_setup/service.rs:1073
16822024-11-13T05:08:52.059ZINFOSledAgent (NexusNotifierTask): Successfully put SledAgentInfo to nexus file = sled-agent/src/nexus.rs:500 new_generation = 1 old_generation = 1 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
16832024-11-13T05:08:52.147ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49540
16842024-11-13T05:08:52.148ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
16852024-11-13T05:08:52.148ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
16862024-11-13T05:08:52.148ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/omicron-zones.json file = common/src/ledger.rs:177
16872024-11-13T05:08:52.150ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/omicron-zones.json file = common/src/ledger.rs:177
16882024-11-13T05:08:52.162ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16892024-11-13T05:08:52.162ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8511034a-a894-457f-9cbc-98e6461f9f5a
16902024-11-13T05:08:52.184ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16912024-11-13T05:08:52.184ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c867bbf7-921c-4db1-887a-4002bcc6b446
16922024-11-13T05:08:52.207ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16932024-11-13T05:08:52.207ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_204f40c4-4e09-4726-a368-a3da2ac1b831
16942024-11-13T05:08:52.228ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16952024-11-13T05:08:52.228ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8e821c87-2d05-4e93-b160-49a645a177c5
16962024-11-13T05:08:52.252ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16972024-11-13T05:08:52.252ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_e770fce0-3dd5-4678-9037-68ea781dbb30
16982024-11-13T05:08:52.272ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
16992024-11-13T05:08:52.272ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c2ce9d45-3d62-4234-a9db-c1d5c8884adc
17002024-11-13T05:08:52.294ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17012024-11-13T05:08:52.294ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c9fdd7dd-67c5-4597-a857-03bc9769dc87
17022024-11-13T05:08:52.315ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17032024-11-13T05:08:52.315ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_27d5a04d-ccb1-4219-851b-abc94499936f
17042024-11-13T05:08:52.336ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17052024-11-13T05:08:52.336ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c7851ba0-6692-4564-8a56-a39ae4767989
17062024-11-13T05:08:52.348ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 200187 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49540 req_id = 2761e7d8-f3d6-4f39-9c61-b6d2fd53f10e response_code = 200 uri = /inventory
17072024-11-13T05:08:54.153ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:39414
17082024-11-13T05:08:54.153ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 43 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:39414 req_id = e9ed9ca5-3e2a-4c09-8843-95612101d9ce response_code = 200 uri = /baseboard
17092024-11-13T05:08:55.076ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52992
17102024-11-13T05:08:55.077ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 150e1804-b3f0-4f2c-82cf-fb2bbaab0984, kind: Service { id: 038e4c2b-2408-42f1-b025-c88f1167faaf }, name: Name("external-dns-038e4c2b-2408-42f1-b025-c88f1167faaf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 226, 83])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b98af94a-ceb1-41e9-82a5-9be8a66b4c85, kind: Service { id: eb4a98a4-3147-445f-9881-33f4dffeca66 }, name: Name("external-dns-eb4a98a4-3147-445f-9881-33f4dffeca66"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 145])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55f88ed5-95d7-43db-b06a-514b7adb4cdb, kind: Service { id: 7641195e-73b4-4972-ad51-4fcacb36a4be }, name: Name("nexus-7641195e-73b4-4972-ad51-4fcacb36a4be"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 204])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 984220cf-9c9a-412c-a465-3579565ca87a, kind: Service { id: e181617d-520a-429d-85ef-385f6dde38ea }, name: Name("nexus-e181617d-520a-429d-85ef-385f6dde38ea"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 83])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9df31eec-63bd-49fd-99e9-b3a05e952553, kind: Service { id: 6c4893ca-1b18-4910-8b18-504e5fcb8dc5 }, name: Name("nexus-6c4893ca-1b18-4910-8b18-504e5fcb8dc5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 5])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
17112024-11-13T05:08:55.077ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17122024-11-13T05:08:55.077ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
17132024-11-13T05:08:55.078ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17142024-11-13T05:08:55.078ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17152024-11-13T05:08:55.078ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17162024-11-13T05:08:55.078ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17172024-11-13T05:08:55.078ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1944 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52992 req_id = 9429fae1-4032-49e3-9cc2-3398becf2f2d response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17182024-11-13T05:08:55.079ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:1083
17192024-11-13T05:08:55.334ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41684
17202024-11-13T05:08:55.334ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 150e1804-b3f0-4f2c-82cf-fb2bbaab0984, kind: Service { id: 038e4c2b-2408-42f1-b025-c88f1167faaf }, name: Name("external-dns-038e4c2b-2408-42f1-b025-c88f1167faaf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 226, 83])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b98af94a-ceb1-41e9-82a5-9be8a66b4c85, kind: Service { id: eb4a98a4-3147-445f-9881-33f4dffeca66 }, name: Name("external-dns-eb4a98a4-3147-445f-9881-33f4dffeca66"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 145])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55f88ed5-95d7-43db-b06a-514b7adb4cdb, kind: Service { id: 7641195e-73b4-4972-ad51-4fcacb36a4be }, name: Name("nexus-7641195e-73b4-4972-ad51-4fcacb36a4be"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 204])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 984220cf-9c9a-412c-a465-3579565ca87a, kind: Service { id: e181617d-520a-429d-85ef-385f6dde38ea }, name: Name("nexus-e181617d-520a-429d-85ef-385f6dde38ea"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 83])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9df31eec-63bd-49fd-99e9-b3a05e952553, kind: Service { id: 6c4893ca-1b18-4910-8b18-504e5fcb8dc5 }, name: Name("nexus-6c4893ca-1b18-4910-8b18-504e5fcb8dc5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 5])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
17212024-11-13T05:08:55.334ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17222024-11-13T05:08:55.334ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
17232024-11-13T05:08:55.334ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17242024-11-13T05:08:55.334ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17252024-11-13T05:08:55.334ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17262024-11-13T05:08:55.334ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17272024-11-13T05:08:55.334ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 281 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:41684 req_id = c73f9c2d-7000-4142-82cd-02ed86dacd30 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17282024-11-13T05:08:55.337ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51500
17292024-11-13T05:08:55.337ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
17302024-11-13T05:08:55.337ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
17312024-11-13T05:08:55.337ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/omicron-zones.json file = common/src/ledger.rs:177
17322024-11-13T05:08:55.337ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/omicron-zones.json file = common/src/ledger.rs:177
17332024-11-13T05:08:55.350ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17342024-11-13T05:08:55.350ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8511034a-a894-457f-9cbc-98e6461f9f5a
17352024-11-13T05:08:55.375ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17362024-11-13T05:08:55.375ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c867bbf7-921c-4db1-887a-4002bcc6b446
17372024-11-13T05:08:55.397ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17382024-11-13T05:08:55.397ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_204f40c4-4e09-4726-a368-a3da2ac1b831
17392024-11-13T05:08:55.420ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17402024-11-13T05:08:55.420ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8e821c87-2d05-4e93-b160-49a645a177c5
17412024-11-13T05:08:55.443ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17422024-11-13T05:08:55.443ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_e770fce0-3dd5-4678-9037-68ea781dbb30
17432024-11-13T05:08:55.463ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17442024-11-13T05:08:55.463ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c2ce9d45-3d62-4234-a9db-c1d5c8884adc
17452024-11-13T05:08:55.484ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17462024-11-13T05:08:55.484ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c9fdd7dd-67c5-4597-a857-03bc9769dc87
17472024-11-13T05:08:55.504ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17482024-11-13T05:08:55.504ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_27d5a04d-ccb1-4219-851b-abc94499936f
17492024-11-13T05:08:55.522ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
17502024-11-13T05:08:55.522ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c7851ba0-6692-4564-8a56-a39ae4767989
17512024-11-13T05:08:55.534ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 197002 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51500 req_id = f876e9a6-ceaa-421f-a643-fc717db477a9 response_code = 200 uri = /inventory
17522024-11-13T05:08:56.241ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47763
17532024-11-13T05:08:56.241ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 150e1804-b3f0-4f2c-82cf-fb2bbaab0984, kind: Service { id: 038e4c2b-2408-42f1-b025-c88f1167faaf }, name: Name("external-dns-038e4c2b-2408-42f1-b025-c88f1167faaf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 226, 83])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b98af94a-ceb1-41e9-82a5-9be8a66b4c85, kind: Service { id: eb4a98a4-3147-445f-9881-33f4dffeca66 }, name: Name("external-dns-eb4a98a4-3147-445f-9881-33f4dffeca66"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 145])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55f88ed5-95d7-43db-b06a-514b7adb4cdb, kind: Service { id: 7641195e-73b4-4972-ad51-4fcacb36a4be }, name: Name("nexus-7641195e-73b4-4972-ad51-4fcacb36a4be"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 204])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 984220cf-9c9a-412c-a465-3579565ca87a, kind: Service { id: e181617d-520a-429d-85ef-385f6dde38ea }, name: Name("nexus-e181617d-520a-429d-85ef-385f6dde38ea"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 83])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9df31eec-63bd-49fd-99e9-b3a05e952553, kind: Service { id: 6c4893ca-1b18-4910-8b18-504e5fcb8dc5 }, name: Name("nexus-6c4893ca-1b18-4910-8b18-504e5fcb8dc5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 5])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
17542024-11-13T05:08:56.241ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17552024-11-13T05:08:56.241ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
17562024-11-13T05:08:56.241ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17572024-11-13T05:08:56.241ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17582024-11-13T05:08:56.241ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17592024-11-13T05:08:56.241ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17602024-11-13T05:08:56.241ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 316 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:47763 req_id = 1391c292-57ff-4351-8c8a-5b327d46eb31 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17612024-11-13T05:08:56.390ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51155
17622024-11-13T05:08:56.391ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 150e1804-b3f0-4f2c-82cf-fb2bbaab0984, kind: Service { id: 038e4c2b-2408-42f1-b025-c88f1167faaf }, name: Name("external-dns-038e4c2b-2408-42f1-b025-c88f1167faaf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 226, 83])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b98af94a-ceb1-41e9-82a5-9be8a66b4c85, kind: Service { id: eb4a98a4-3147-445f-9881-33f4dffeca66 }, name: Name("external-dns-eb4a98a4-3147-445f-9881-33f4dffeca66"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 145])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55f88ed5-95d7-43db-b06a-514b7adb4cdb, kind: Service { id: 7641195e-73b4-4972-ad51-4fcacb36a4be }, name: Name("nexus-7641195e-73b4-4972-ad51-4fcacb36a4be"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 204])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 984220cf-9c9a-412c-a465-3579565ca87a, kind: Service { id: e181617d-520a-429d-85ef-385f6dde38ea }, name: Name("nexus-e181617d-520a-429d-85ef-385f6dde38ea"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 83])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9df31eec-63bd-49fd-99e9-b3a05e952553, kind: Service { id: 6c4893ca-1b18-4910-8b18-504e5fcb8dc5 }, name: Name("nexus-6c4893ca-1b18-4910-8b18-504e5fcb8dc5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 5])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
17632024-11-13T05:08:56.391ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17642024-11-13T05:08:56.391ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
17652024-11-13T05:08:56.391ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17662024-11-13T05:08:56.391ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17672024-11-13T05:08:56.391ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
17682024-11-13T05:08:56.391ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
17692024-11-13T05:08:56.391ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 317 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51155 req_id = f20c90d4-ef27-4c82-ae82-4d4e06f317e1 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17702024-11-13T05:09:02.977ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39510
17712024-11-13T05:09:02.978ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1124 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39510 req_id = 76621efa-17c0-4fac-bc81-408b3554201a response_code = 200 uri = /vpc-routes
17722024-11-13T05:09:03.108ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
17732024-11-13T05:09:03.108ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:39510 req_id = 184e3f56-1f59-4387-ac42-21f394c68d29 response_code = 204 uri = /eip-gateways
17742024-11-13T05:09:03.205ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 1857 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:39510 req_id = b249fd47-f1d6-4cda-9535-8df862021bf5 response_code = 204 uri = /vpc-routes
17752024-11-13T05:09:10.321ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39033
17762024-11-13T05:09:10.321ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 259 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39033 req_id = 5e4cd25d-8d0b-4332-9379-8f182d39fe3f response_code = 200 uri = /v2p
17772024-11-13T05:09:10.363ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57615
17782024-11-13T05:09:10.364ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 28 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57615 req_id = c1673e29-2c5e-421f-baa0-cb6523e6723b response_code = 200 uri = /vpc-routes
17792024-11-13T05:09:10.412ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
17802024-11-13T05:09:10.412ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57615 req_id = 38920315-25a4-4732-bc4f-713bccdce477 response_code = 204 uri = /eip-gateways
17812024-11-13T05:09:10.422ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 219 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57615 req_id = 6014c694-baa5-4e96-b5eb-b257a96e8d6a response_code = 204 uri = /vpc-routes
17822024-11-13T05:09:10.458ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63083
17832024-11-13T05:09:10.459ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
17842024-11-13T05:09:10.535ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
17852024-11-13T05:09:10.601ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 142496 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:63083 req_id = c540a499-727e-4432-9ab2-c11a3a5f70ae response_code = 204 uri = /switch-ports
17862024-11-13T05:09:10.606ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 2786 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63083 req_id = 23a7bc7b-18af-4d79-bbac-9cd1152d86dc response_code = 200 uri = /network-bootstore-config
17872024-11-13T05:09:10.646ZINFOSledAgent (bootstore): Attempting to update network config with generation: 2, current_generation: 1 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
17882024-11-13T05:09:10.646ZINFOSledAgent (bootstore): Writing ledger to /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
17892024-11-13T05:09:10.647ZINFOSledAgent (bootstore): Writing ledger to /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
17902024-11-13T05:09:10.649ZINFOSledAgent (bootstore): Broadcasting network config with generation 2 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
17912024-11-13T05:09:10.649ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 3033 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:63083 req_id = cef9a8fe-3a2d-44db-a834-830cfdc5ca04 response_code = 204 uri = /network-bootstore-config
17922024-11-13T05:09:11.142ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51955
17932024-11-13T05:09:11.142ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51955 req_id = 4d1ef8fb-3f7b-438e-9ea5-d4968a6306ee response_code = 200 uri = /vpc-routes
17942024-11-13T05:09:11.179ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
17952024-11-13T05:09:11.179ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51955 req_id = 8f9ea461-69e4-44b7-94fc-8c44e35a5bb4 response_code = 204 uri = /eip-gateways
17962024-11-13T05:09:11.199ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 357 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51955 req_id = 9be7640e-f0cd-400d-ae21-9ff1a4ca2c78 response_code = 204 uri = /vpc-routes
17972024-11-13T05:09:11.228ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35425
17982024-11-13T05:09:11.228ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35425 req_id = 1e27534b-8281-4c05-8b2b-3b741ab10f7f response_code = 200 uri = /vpc-routes
17992024-11-13T05:09:11.252ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
18002024-11-13T05:09:11.252ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35425 req_id = 3d94763d-b193-46f4-b521-f76334c242ff response_code = 204 uri = /eip-gateways
18012024-11-13T05:09:11.264ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 262 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35425 req_id = 0b97aa4f-a737-4f4f-8c27-f78c4c241eeb response_code = 204 uri = /vpc-routes
18022024-11-13T05:09:11.360ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63338
18032024-11-13T05:09:11.360ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63338 req_id = 89698233-9218-4bc7-abf6-c38874895604 response_code = 200 uri = /v2p
18042024-11-13T05:09:11.444ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63122
18052024-11-13T05:09:11.444ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
18062024-11-13T05:09:11.482ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
18072024-11-13T05:09:11.538ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 94266 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:63122 req_id = 354bdc47-7f57-4c3c-be96-6ae4861ec03a response_code = 204 uri = /switch-ports
18082024-11-13T05:09:11.538ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38765
18092024-11-13T05:09:11.538ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 147 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38765 req_id = 823dbc4e-1588-44b0-bd14-27cd6b3ce841 response_code = 200 uri = /v2p
18102024-11-13T05:09:11.540ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63122 req_id = 881888a0-10d7-4e42-b782-4e0b32f84965 response_code = 200 uri = /network-bootstore-config
18112024-11-13T05:09:11.556ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43978
18122024-11-13T05:09:11.557ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
18132024-11-13T05:09:11.592ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
18142024-11-13T05:09:11.647ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 90484 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:43978 req_id = 3589d983-ab41-4b46-a0f6-e279f2ebd79a response_code = 204 uri = /switch-ports
18152024-11-13T05:09:11.649ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43978 req_id = 55b67e1f-2635-4537-881b-133eb1c46e78 response_code = 200 uri = /network-bootstore-config
18162024-11-13T05:09:24.187ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:57710
18172024-11-13T05:09:24.187ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 30 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:57710 req_id = 1982ec6d-2bfd-480e-bd80-0a6372279103 response_code = 200 uri = /baseboard
18182024-11-13T05:09:40.095ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58863
18192024-11-13T05:09:40.095ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58863 req_id = 83eb0263-dd22-4fb6-b8b7-261feb3f5341 response_code = 200 uri = /vpc-routes
18202024-11-13T05:09:40.167ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
18212024-11-13T05:09:40.167ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58863 req_id = b6cc1d27-35b2-41ff-953b-c00acbbb270c response_code = 204 uri = /eip-gateways
18222024-11-13T05:09:40.178ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 190 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58863 req_id = 44af3eed-8556-4f14-932c-5c0a611c4415 response_code = 204 uri = /vpc-routes
18232024-11-13T05:09:40.206ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44738
18242024-11-13T05:09:40.206ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
18252024-11-13T05:09:40.243ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
18262024-11-13T05:09:40.298ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 91939 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:44738 req_id = b505d9e2-c9fa-48a1-b606-be28f333ea6f response_code = 204 uri = /switch-ports
18272024-11-13T05:09:40.298ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44738 req_id = ab44f701-6365-4d82-9af7-94af37d9e85d response_code = 200 uri = /network-bootstore-config
18282024-11-13T05:09:40.320ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64679
18292024-11-13T05:09:40.321ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 187 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64679 req_id = 3d18ea19-e311-428a-a101-fe2f0204286b response_code = 200 uri = /v2p
18302024-11-13T05:09:41.023ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60367
18312024-11-13T05:09:41.023ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60367 req_id = c02ceb71-c3ab-4254-b651-18cf1cad36da response_code = 200 uri = /vpc-routes
18322024-11-13T05:09:41.052ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
18332024-11-13T05:09:41.052ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60367 req_id = 381ccb7a-c57a-48ed-bc1c-0ef271626220 response_code = 204 uri = /eip-gateways
18342024-11-13T05:09:41.066ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60731
18352024-11-13T05:09:41.066ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60731 req_id = 52b773eb-a62e-4a8e-91c7-bea18657b829 response_code = 200 uri = /v2p
18362024-11-13T05:09:41.081ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 237 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60367 req_id = 96610f43-f405-41a4-bc87-3f961fb6a4e6 response_code = 204 uri = /vpc-routes
18372024-11-13T05:09:41.167ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51577
18382024-11-13T05:09:41.167ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
18392024-11-13T05:09:41.203ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
18402024-11-13T05:09:41.257ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 90276 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:51577 req_id = 9e4981bc-239d-463d-b6cd-a35b8417c340 response_code = 204 uri = /switch-ports
18412024-11-13T05:09:41.257ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33242
18422024-11-13T05:09:41.258ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47903
18432024-11-13T05:09:41.258ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47903 req_id = 746b7a42-2a47-46a4-acfc-3617de121c39 response_code = 200 uri = /vpc-routes
18442024-11-13T05:09:41.258ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 247 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33242 req_id = 4476f77b-c74b-45c8-89c3-e51b9671f90a response_code = 200 uri = /v2p
18452024-11-13T05:09:41.258ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 264 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51577 req_id = d8e0f904-1502-475b-b926-395ca42f2882 response_code = 200 uri = /network-bootstore-config
18462024-11-13T05:09:41.275ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
18472024-11-13T05:09:41.275ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 66 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47903 req_id = 72657609-3a33-4e44-aa0c-3c80fcf6939d response_code = 204 uri = /eip-gateways
18482024-11-13T05:09:41.285ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 259 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47903 req_id = 792fb6c4-1ba4-4201-b455-9d59f841dae0 response_code = 204 uri = /vpc-routes
18492024-11-13T05:09:41.322ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41038
18502024-11-13T05:09:41.322ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
18512024-11-13T05:09:41.359ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
18522024-11-13T05:09:41.413ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 90264 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:41038 req_id = f84f493f-ae24-4729-85b9-7103ce4af86b response_code = 204 uri = /switch-ports
18532024-11-13T05:09:41.413ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41038 req_id = d6ce50c1-9b51-45de-b034-fa577a951798 response_code = 200 uri = /network-bootstore-config
18542024-11-13T05:09:54.220ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:56786
18552024-11-13T05:09:54.220ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 31 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:56786 req_id = 56991582-d90b-49e6-b832-a5a11f4ee14c response_code = 200 uri = /baseboard
18562024-11-13T05:10:10.191ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51927
18572024-11-13T05:10:10.191ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51927 req_id = 486e9aa5-1ad4-4773-903e-b6939150e2db response_code = 200 uri = /vpc-routes
18582024-11-13T05:10:10.238ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
18592024-11-13T05:10:10.238ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51927 req_id = ed50fc3d-4ba2-4192-959e-b89e0f76f9a4 response_code = 204 uri = /eip-gateways
18602024-11-13T05:10:10.249ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51927 req_id = e3cc8afb-d20b-45f8-9dc1-5594be59298f response_code = 204 uri = /vpc-routes
18612024-11-13T05:10:10.305ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33769
18622024-11-13T05:10:10.305ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33769 req_id = 7b4995c3-5a9e-48eb-bc24-983fc9cdf668 response_code = 200 uri = /v2p
18632024-11-13T05:10:10.310ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47067
18642024-11-13T05:10:10.310ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
18652024-11-13T05:10:10.349ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
18662024-11-13T05:10:10.403ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 93000 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:47067 req_id = ac693b66-eece-4114-b47e-f6e4d34feb3a response_code = 204 uri = /switch-ports
18672024-11-13T05:10:10.403ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47067 req_id = 48588685-5c7d-440e-a1c5-f5684ff3ce6f response_code = 200 uri = /network-bootstore-config
18682024-11-13T05:10:11.076ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52279
18692024-11-13T05:10:11.076ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52279 req_id = 48861ff6-96a9-4800-b144-47cc8ee95efe response_code = 200 uri = /vpc-routes
18702024-11-13T05:10:11.139ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
18712024-11-13T05:10:11.139ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 120 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52279 req_id = d0b5950d-8667-4562-bfbc-07acd63e0df7 response_code = 204 uri = /eip-gateways
18722024-11-13T05:10:11.152ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 233 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52279 req_id = 270907a8-f99e-4eae-8d74-6aecbdaeeee4 response_code = 204 uri = /vpc-routes
18732024-11-13T05:10:11.237ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43828
18742024-11-13T05:10:11.237ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
18752024-11-13T05:10:11.279ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
18762024-11-13T05:10:11.334ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 96875 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:43828 req_id = 3eb375d3-3369-4863-84c0-3b4e80ba0111 response_code = 204 uri = /switch-ports
18772024-11-13T05:10:11.334ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47460
18782024-11-13T05:10:11.334ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40574
18792024-11-13T05:10:11.334ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34248
18802024-11-13T05:10:11.335ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 277 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34248 req_id = f9f3ca40-8ecd-42b2-ac60-072d943acc93 response_code = 200 uri = /v2p
18812024-11-13T05:10:11.335ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 274 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43828 req_id = 946c5e3a-a101-4418-a1a0-10091ad82587 response_code = 200 uri = /network-bootstore-config
18822024-11-13T05:10:11.335ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 319 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40574 req_id = fb0b1190-e762-4b70-924e-3b2e58e84089 response_code = 200 uri = /v2p
18832024-11-13T05:10:11.335ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47460 req_id = aa320530-639d-4d19-b785-bf535b44a403 response_code = 200 uri = /vpc-routes
18842024-11-13T05:10:11.380ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
18852024-11-13T05:10:11.380ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47460 req_id = 3401920d-7d74-4709-ae6c-b6c6b07266db response_code = 204 uri = /eip-gateways
18862024-11-13T05:10:11.390ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 283 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47460 req_id = d3cd2285-c340-456f-ae72-9db008886dd7 response_code = 204 uri = /vpc-routes
18872024-11-13T05:10:11.400ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62109
18882024-11-13T05:10:11.400ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
18892024-11-13T05:10:11.438ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
18902024-11-13T05:10:11.492ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 92666 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:62109 req_id = f1d1150b-5ecb-4f70-ada1-25c996a9f066 response_code = 204 uri = /switch-ports
18912024-11-13T05:10:11.493ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62109 req_id = fefaac5a-e357-4d33-9744-c5725411fd92 response_code = 200 uri = /network-bootstore-config
18922024-11-13T05:10:24.255ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:43451
18932024-11-13T05:10:24.255ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 82 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:43451 req_id = 076ab9b9-9d0d-41eb-a3c2-4efae177af8d response_code = 200 uri = /baseboard
18942024-11-13T05:10:40.057ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39147
18952024-11-13T05:10:40.057ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39147 req_id = ede46c2a-fe44-4d51-a32a-066093483f24 response_code = 200 uri = /vpc-routes
18962024-11-13T05:10:40.104ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
18972024-11-13T05:10:40.104ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:39147 req_id = ee8d8e80-3293-4115-989b-3c6d3f4d7839 response_code = 204 uri = /eip-gateways
18982024-11-13T05:10:40.113ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 174 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:39147 req_id = f6fdbf79-0257-4d0b-9891-5126aebb9dab response_code = 204 uri = /vpc-routes
18992024-11-13T05:10:40.179ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50921
19002024-11-13T05:10:40.179ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19012024-11-13T05:10:40.213ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19022024-11-13T05:10:40.270ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 90987 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:50921 req_id = 23af8186-95cd-4916-8c13-4ee9898a3b96 response_code = 204 uri = /switch-ports
19032024-11-13T05:10:40.270ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50921 req_id = ee27b269-2fdb-4964-8f30-34b0ca17e4a2 response_code = 200 uri = /network-bootstore-config
19042024-11-13T05:10:40.283ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63693
19052024-11-13T05:10:40.283ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 157 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63693 req_id = fd2037e9-53d8-4dfc-a9f8-dd1966a13b54 response_code = 200 uri = /v2p
19062024-11-13T05:10:41.018ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42309
19072024-11-13T05:10:41.018ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42309 req_id = fe9db4de-d29b-4968-a08d-cf6d0b50f9e9 response_code = 200 uri = /vpc-routes
19082024-11-13T05:10:41.064ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
19092024-11-13T05:10:41.064ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42309 req_id = 4907ebc0-9177-4c63-bd7e-4f866728e0eb response_code = 204 uri = /eip-gateways
19102024-11-13T05:10:41.075ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 263 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42309 req_id = 321f687a-0958-4ee5-87f3-909af4cd6694 response_code = 204 uri = /vpc-routes
19112024-11-13T05:10:41.132ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63043
19122024-11-13T05:10:41.132ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63043 req_id = f5676c27-2024-4294-9a1e-a1bd28814cb0 response_code = 200 uri = /v2p
19132024-11-13T05:10:41.165ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33627
19142024-11-13T05:10:41.165ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 50 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33627 req_id = 9099cbd3-7d93-467b-a3e6-fcb2b2604c53 response_code = 200 uri = /vpc-routes
19152024-11-13T05:10:41.170ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47504
19162024-11-13T05:10:41.170ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19172024-11-13T05:10:41.212ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19182024-11-13T05:10:41.273ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 102371 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:47504 req_id = faf886c9-d1e9-44ac-8a4c-d8be7a26dba3 response_code = 204 uri = /switch-ports
19192024-11-13T05:10:41.273ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
19202024-11-13T05:10:41.273ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:33627 req_id = e1b51af6-b6a1-4051-9c7e-8edba7d16b85 response_code = 204 uri = /eip-gateways
19212024-11-13T05:10:41.273ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47504 req_id = 8cf40ce6-4a36-4f91-bfca-02d9c330023d response_code = 200 uri = /network-bootstore-config
19222024-11-13T05:10:41.297ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61658
19232024-11-13T05:10:41.297ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61658 req_id = 622fb070-026b-4a16-a816-85422556f0d4 response_code = 200 uri = /v2p
19242024-11-13T05:10:41.300ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45094
19252024-11-13T05:10:41.300ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19262024-11-13T05:10:41.334ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19272024-11-13T05:10:41.391ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 91345 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:45094 req_id = 38c143db-3492-4c2c-8d14-787fff3f0c7f response_code = 204 uri = /switch-ports
19282024-11-13T05:10:41.392ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 263 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45094 req_id = 84e928cc-df8f-4a31-ad1f-d33faeec7d71 response_code = 200 uri = /network-bootstore-config
19292024-11-13T05:10:41.392ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 408 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:33627 req_id = 31f53fba-1d37-45da-875c-224d92161997 response_code = 204 uri = /vpc-routes
19302024-11-13T05:10:54.300ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47344
19312024-11-13T05:10:54.302ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 33 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47344 req_id = 674c6004-07b3-4ae7-af86-b820f9ac6f96 response_code = 200 uri = /baseboard
19322024-11-13T05:11:10.404ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34480
19332024-11-13T05:11:10.404ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34480 req_id = 2930054e-5af8-4557-b1c8-1be2cc3926d0 response_code = 200 uri = /vpc-routes
19342024-11-13T05:11:10.446ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
19352024-11-13T05:11:10.447ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:34480 req_id = ecad827b-8d13-4ac9-bd1e-04d773030a94 response_code = 204 uri = /eip-gateways
19362024-11-13T05:11:10.494ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49733
19372024-11-13T05:11:10.494ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49733 req_id = fc63b8d3-0d89-42f3-b4c1-b7a0e96cf837 response_code = 200 uri = /v2p
19382024-11-13T05:11:10.500ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 186 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:34480 req_id = 9b98835c-d207-475a-96bc-a9eff7a891d8 response_code = 204 uri = /vpc-routes
19392024-11-13T05:11:10.545ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38234
19402024-11-13T05:11:10.545ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19412024-11-13T05:11:10.588ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19422024-11-13T05:11:10.644ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 99704 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:38234 req_id = 7d2853ff-2d7d-4a8c-8096-09ea97bec09a response_code = 204 uri = /switch-ports
19432024-11-13T05:11:10.645ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38234 req_id = f8323d3a-8c5e-4efe-bd4a-b931fe681988 response_code = 200 uri = /network-bootstore-config
19442024-11-13T05:11:11.290ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44012
19452024-11-13T05:11:11.290ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44012 req_id = f26298a2-5cb1-4a38-bbb9-ffd5a3f3916c response_code = 200 uri = /vpc-routes
19462024-11-13T05:11:11.336ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
19472024-11-13T05:11:11.336ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44012 req_id = 90631042-efe2-4f13-a641-80dddc743e61 response_code = 204 uri = /eip-gateways
19482024-11-13T05:11:11.345ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 228 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44012 req_id = 003813ce-9ab3-4982-af66-d1030518f0c0 response_code = 204 uri = /vpc-routes
19492024-11-13T05:11:11.404ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57698
19502024-11-13T05:11:11.404ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57698 req_id = d97c8d10-3046-40dd-ba04-50bb2ecbbf72 response_code = 200 uri = /vpc-routes
19512024-11-13T05:11:11.406ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58427
19522024-11-13T05:11:11.406ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 138 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58427 req_id = 7c43f59f-95cb-45b2-8e4c-b659abf21d11 response_code = 200 uri = /v2p
19532024-11-13T05:11:11.439ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56104
19542024-11-13T05:11:11.439ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56104 req_id = 5268ee57-eebe-40d2-9671-793d515bbf86 response_code = 200 uri = /v2p
19552024-11-13T05:11:11.450ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
19562024-11-13T05:11:11.450ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57698 req_id = 3954407e-5e69-45d0-b1a4-e3ab66a16ea4 response_code = 204 uri = /eip-gateways
19572024-11-13T05:11:11.460ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 253 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57698 req_id = 4f1b5f75-1501-4a9b-bf0b-c542cb146f15 response_code = 204 uri = /vpc-routes
19582024-11-13T05:11:11.460ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50956
19592024-11-13T05:11:11.461ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19602024-11-13T05:11:11.499ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19612024-11-13T05:11:11.554ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 93085 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:50956 req_id = a73cba8a-4883-4b18-a116-8b70422d0c44 response_code = 204 uri = /switch-ports
19622024-11-13T05:11:11.554ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 156 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50956 req_id = 4fcf385d-9c5c-4aa7-b3e1-b5404d7951af response_code = 200 uri = /network-bootstore-config
19632024-11-13T05:11:11.619ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46274
19642024-11-13T05:11:11.619ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19652024-11-13T05:11:11.651ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19662024-11-13T05:11:11.704ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 85136 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:46274 req_id = c045d85e-1354-49d0-91c5-32a189aad045 response_code = 204 uri = /switch-ports
19672024-11-13T05:11:11.704ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46274 req_id = d55db0f3-38fa-4869-9d8e-deb57ae38eed response_code = 200 uri = /network-bootstore-config
19682024-11-13T05:11:18.063ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
19692024-11-13T05:11:24.334ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:60115
19702024-11-13T05:11:24.334ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 48 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:60115 req_id = 7622c68f-1347-4ce9-96d6-2ab2b716f6d2 response_code = 200 uri = /baseboard
19712024-11-13T05:11:40.133ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51234
19722024-11-13T05:11:40.133ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 54 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51234 req_id = 11ce1908-528c-4ab9-a783-14e687dff356 response_code = 200 uri = /vpc-routes
19732024-11-13T05:11:40.151ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
19742024-11-13T05:11:40.151ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 77 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51234 req_id = 06a073e2-bfb2-4bbf-98d7-b57a048dedc1 response_code = 204 uri = /eip-gateways
19752024-11-13T05:11:40.159ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 196 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51234 req_id = 8da499dd-1b18-4eb2-91cc-cf1af479469c response_code = 204 uri = /vpc-routes
19762024-11-13T05:11:40.229ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56360
19772024-11-13T05:11:40.230ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56360 req_id = ce08149d-3846-4563-a82a-8af77f3282c9 response_code = 200 uri = /v2p
19782024-11-13T05:11:40.286ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34128
19792024-11-13T05:11:40.286ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19802024-11-13T05:11:40.324ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19812024-11-13T05:11:40.380ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 94047 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:34128 req_id = 745d49be-d568-4595-ab15-ec6829887ec2 response_code = 204 uri = /switch-ports
19822024-11-13T05:11:40.380ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34128 req_id = 179d7c48-903b-498d-888c-2a6eff0115b4 response_code = 200 uri = /network-bootstore-config
19832024-11-13T05:11:40.995ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37736
19842024-11-13T05:11:40.995ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37736 req_id = f8bdf5ca-9fdc-436e-a61a-2a587517f151 response_code = 200 uri = /vpc-routes
19852024-11-13T05:11:41.059ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
19862024-11-13T05:11:41.059ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37736 req_id = bb285575-5d25-40fa-a09c-4aac8d71ff50 response_code = 204 uri = /eip-gateways
19872024-11-13T05:11:41.070ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 203 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37736 req_id = 5e4e2de3-a12b-44df-bcfd-401160d713ca response_code = 204 uri = /vpc-routes
19882024-11-13T05:11:41.136ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51516
19892024-11-13T05:11:41.136ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19902024-11-13T05:11:41.173ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19912024-11-13T05:11:41.229ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 93004 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:51516 req_id = 10f3621c-4efa-419d-bd8d-60497c9b8f11 response_code = 204 uri = /switch-ports
19922024-11-13T05:11:41.229ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42354
19932024-11-13T05:11:41.229ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 193 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42354 req_id = cbebe09f-1112-47f3-9ab6-3673c621035d response_code = 200 uri = /v2p
19942024-11-13T05:11:41.229ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 201 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51516 req_id = 43fe31f5-92b5-4b90-a42c-c0c1a627737a response_code = 200 uri = /network-bootstore-config
19952024-11-13T05:11:41.264ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53621
19962024-11-13T05:11:41.265ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53621 req_id = ac536fb5-33fb-4a58-8462-9cc2a9f8a854 response_code = 200 uri = /vpc-routes
19972024-11-13T05:11:41.278ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
19982024-11-13T05:11:41.278ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53621 req_id = 01c1b71c-3015-4cae-b601-34bc81bee52e response_code = 204 uri = /eip-gateways
19992024-11-13T05:11:41.286ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 253 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53621 req_id = 0a12bf13-cb24-4405-b0ab-0f589c89911e response_code = 204 uri = /vpc-routes
20002024-11-13T05:11:41.377ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63561
20012024-11-13T05:11:41.378ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63561 req_id = 0d8ad8d2-74d8-41cc-86c7-38af343949a6 response_code = 200 uri = /v2p
20022024-11-13T05:11:41.389ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54694
20032024-11-13T05:11:41.389ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20042024-11-13T05:11:41.428ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20052024-11-13T05:11:41.485ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 96116 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:54694 req_id = a913cdaf-9203-4a86-a45a-853cc4a9e6ab response_code = 204 uri = /switch-ports
20062024-11-13T05:11:41.485ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54694 req_id = c7d2a3a5-39cf-48b3-a33e-aa187e36ca13 response_code = 200 uri = /network-bootstore-config
20072024-11-13T05:11:54.367ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:41879
20082024-11-13T05:11:54.367ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 30 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:41879 req_id = 3e766e05-721a-417a-be65-864fffb8ea5a response_code = 200 uri = /baseboard
20092024-11-13T05:12:10.262ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40853
20102024-11-13T05:12:10.262ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 138 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40853 req_id = c5a6a494-c01a-4a35-9c13-6bc48bcfe0e9 response_code = 200 uri = /v2p
20112024-11-13T05:12:10.359ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57447
20122024-11-13T05:12:10.359ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 601 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57447 req_id = b0369172-2794-432c-940f-d6641ebf0f4b response_code = 200 uri = /vpc-routes
20132024-11-13T05:12:10.406ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
20142024-11-13T05:12:10.406ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57447 req_id = 8c824923-d25c-40c5-8379-4785f78fba47 response_code = 204 uri = /eip-gateways
20152024-11-13T05:12:10.416ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 252 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57447 req_id = 6db338ed-fda8-4e1e-bac5-c5687bb1cbe7 response_code = 204 uri = /vpc-routes
20162024-11-13T05:12:10.438ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56568
20172024-11-13T05:12:10.438ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20182024-11-13T05:12:10.475ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20192024-11-13T05:12:10.533ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 95180 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:56568 req_id = a14317ec-73c8-461d-b8f9-5b4d644ec3c2 response_code = 204 uri = /switch-ports
20202024-11-13T05:12:10.533ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56568 req_id = c637f6ea-7ceb-4852-9895-0ab609f69a9e response_code = 200 uri = /network-bootstore-config
20212024-11-13T05:12:11.121ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62534
20222024-11-13T05:12:11.121ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62534 req_id = c677f1e4-447d-405d-a3d6-8b58db2dc7ce response_code = 200 uri = /vpc-routes
20232024-11-13T05:12:11.141ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
20242024-11-13T05:12:11.141ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 308 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:62534 req_id = 7d615b38-2255-4776-9807-a053c4473523 response_code = 204 uri = /eip-gateways
20252024-11-13T05:12:11.152ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 274 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:62534 req_id = 8f45b48a-dec2-47e4-8208-017fcf8225a4 response_code = 204 uri = /vpc-routes
20262024-11-13T05:12:11.198ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54236
20272024-11-13T05:12:11.198ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54236 req_id = 734452b4-ab04-48ec-9f38-3080af27174a response_code = 200 uri = /vpc-routes
20282024-11-13T05:12:11.245ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
20292024-11-13T05:12:11.245ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54236 req_id = 99254a6f-35b3-4ab0-a8d6-a0cb70e0abe8 response_code = 204 uri = /eip-gateways
20302024-11-13T05:12:11.252ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 194 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54236 req_id = d7512420-53fe-46cd-a609-f92bbf7a95a3 response_code = 204 uri = /vpc-routes
20312024-11-13T05:12:11.340ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53545
20322024-11-13T05:12:11.340ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53545 req_id = 9bfbede8-3fa6-4aac-8296-f81eac5cfd89 response_code = 200 uri = /v2p
20332024-11-13T05:12:11.406ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57270
20342024-11-13T05:12:11.406ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20352024-11-13T05:12:11.446ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20362024-11-13T05:12:11.502ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 95920 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:57270 req_id = d8233431-0467-4069-9091-0b4de428b40f response_code = 204 uri = /switch-ports
20372024-11-13T05:12:11.502ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42106
20382024-11-13T05:12:11.502ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 220 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42106 req_id = c4a9316b-a1b4-4d8b-85ac-aa02624bce8d response_code = 200 uri = /v2p
20392024-11-13T05:12:11.502ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 229 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57270 req_id = abc8ced4-0d9a-445c-bc2e-eedd50080d8e response_code = 200 uri = /network-bootstore-config
20402024-11-13T05:12:11.512ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53503
20412024-11-13T05:12:11.513ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20422024-11-13T05:12:11.546ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20432024-11-13T05:12:11.602ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89466 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:53503 req_id = a5de9fb1-e2e9-4efc-8ac2-15d8865c6a1b response_code = 204 uri = /switch-ports
20442024-11-13T05:12:11.602ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 228 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53503 req_id = 811bb244-252a-4003-b448-630a1918bffc response_code = 200 uri = /network-bootstore-config
20452024-11-13T05:12:24.400ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:35503
20462024-11-13T05:12:24.400ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 43 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:35503 req_id = 60ba306b-26ed-403b-8650-cf3a03a1ae0c response_code = 200 uri = /baseboard
20472024-11-13T05:12:40.076ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53678
20482024-11-13T05:12:40.076ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53678 req_id = a573ca51-65bd-4b4d-b778-451782489741 response_code = 200 uri = /vpc-routes
20492024-11-13T05:12:40.135ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
20502024-11-13T05:12:40.135ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53678 req_id = 79d67599-54b2-42a8-92ea-88895bc4ae84 response_code = 204 uri = /eip-gateways
20512024-11-13T05:12:40.145ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 175 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53678 req_id = 83bebafd-f913-43dc-bb02-a054f45388b0 response_code = 204 uri = /vpc-routes
20522024-11-13T05:12:40.262ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46193
20532024-11-13T05:12:40.262ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 151 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46193 req_id = f0ac162e-b10c-442f-b49d-2207766a223d response_code = 200 uri = /v2p
20542024-11-13T05:12:40.274ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:65507
20552024-11-13T05:12:40.274ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20562024-11-13T05:12:40.312ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20572024-11-13T05:12:40.366ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 92487 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:65507 req_id = 81a4a4d0-932f-458c-9b99-ec00ba365c20 response_code = 204 uri = /switch-ports
20582024-11-13T05:12:40.367ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:65507 req_id = 7071fc22-f9c2-4b0d-b7e8-271434f675ac response_code = 200 uri = /network-bootstore-config
20592024-11-13T05:12:40.914ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39020
20602024-11-13T05:12:40.914ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 73 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39020 req_id = 90f12e6d-4499-43c4-8f3d-5af493652008 response_code = 200 uri = /vpc-routes
20612024-11-13T05:12:40.965ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
20622024-11-13T05:12:40.965ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:39020 req_id = b182b1c1-0535-4a0c-bba7-e98d09314fde response_code = 204 uri = /eip-gateways
20632024-11-13T05:12:40.975ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 227 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:39020 req_id = 8ebf84af-1a0f-45e6-ba70-58c649d87a06 response_code = 204 uri = /vpc-routes
20642024-11-13T05:12:41.071ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57394
20652024-11-13T05:12:41.071ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 66 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57394 req_id = 52ae7658-ba6b-4cea-9bf7-c1e2052c8802 response_code = 200 uri = /vpc-routes
20662024-11-13T05:12:41.124ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
20672024-11-13T05:12:41.124ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57394 req_id = de95d5d6-e49b-43b1-af4c-2622fb16990b response_code = 204 uri = /eip-gateways
20682024-11-13T05:12:41.133ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 219 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57394 req_id = 5591d9d8-05e9-48cf-bd9f-207bf59a25e6 response_code = 204 uri = /vpc-routes
20692024-11-13T05:12:41.228ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42475
20702024-11-13T05:12:41.228ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42475 req_id = 8cdb2549-be85-4317-8122-446fee42e29f response_code = 200 uri = /v2p
20712024-11-13T05:12:41.234ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39863
20722024-11-13T05:12:41.234ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 81 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39863 req_id = 36730cd2-310d-4a33-a1a6-7bfabf1aae1b response_code = 200 uri = /v2p
20732024-11-13T05:12:41.249ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48857
20742024-11-13T05:12:41.249ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20752024-11-13T05:12:41.288ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20762024-11-13T05:12:41.344ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 94957 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:48857 req_id = 39367453-66e4-4b79-9789-4604779a8b02 response_code = 204 uri = /switch-ports
20772024-11-13T05:12:41.344ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36523
20782024-11-13T05:12:41.344ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20792024-11-13T05:12:41.348ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 212 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48857 req_id = 378b066a-7852-488c-833b-7fc68d6ab212 response_code = 200 uri = /network-bootstore-config
20802024-11-13T05:12:41.378ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20812024-11-13T05:12:41.436ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 91998 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:36523 req_id = 4a804eca-8b94-4951-aba9-a792d40e5d9c response_code = 204 uri = /switch-ports
20822024-11-13T05:12:41.436ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 147 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36523 req_id = a4824f31-b76a-4ef0-b17d-c84702759bbd response_code = 200 uri = /network-bootstore-config
20832024-11-13T05:12:54.434ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:41016
20842024-11-13T05:12:54.434ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 30 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:41016 req_id = f0e83a59-186e-4956-8cc9-c8dc6136fb2b response_code = 200 uri = /baseboard
20852024-11-13T05:13:10.374ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60862
20862024-11-13T05:13:10.374ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60862 req_id = cca5a738-3a14-447b-9aec-f37920919199 response_code = 200 uri = /vpc-routes
20872024-11-13T05:13:10.426ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
20882024-11-13T05:13:10.426ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:60862 req_id = 353499bc-4ed7-42b7-87b3-fdbb0c239963 response_code = 204 uri = /eip-gateways
20892024-11-13T05:13:10.435ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 226 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:60862 req_id = 18423adb-b06a-4048-9ccb-eb6b4419b7b4 response_code = 204 uri = /vpc-routes
20902024-11-13T05:13:10.561ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54778
20912024-11-13T05:13:10.561ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54778 req_id = 88297228-b068-4991-91a3-c7f991f449e3 response_code = 200 uri = /v2p
20922024-11-13T05:13:10.614ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59814
20932024-11-13T05:13:10.614ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 150e1804-b3f0-4f2c-82cf-fb2bbaab0984, kind: Service { id: 038e4c2b-2408-42f1-b025-c88f1167faaf }, name: Name("external-dns-038e4c2b-2408-42f1-b025-c88f1167faaf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 226, 83])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b98af94a-ceb1-41e9-82a5-9be8a66b4c85, kind: Service { id: eb4a98a4-3147-445f-9881-33f4dffeca66 }, name: Name("external-dns-eb4a98a4-3147-445f-9881-33f4dffeca66"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 145])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55f88ed5-95d7-43db-b06a-514b7adb4cdb, kind: Service { id: 7641195e-73b4-4972-ad51-4fcacb36a4be }, name: Name("nexus-7641195e-73b4-4972-ad51-4fcacb36a4be"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 204])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 984220cf-9c9a-412c-a465-3579565ca87a, kind: Service { id: e181617d-520a-429d-85ef-385f6dde38ea }, name: Name("nexus-e181617d-520a-429d-85ef-385f6dde38ea"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 83])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9df31eec-63bd-49fd-99e9-b3a05e952553, kind: Service { id: 6c4893ca-1b18-4910-8b18-504e5fcb8dc5 }, name: Name("nexus-6c4893ca-1b18-4910-8b18-504e5fcb8dc5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 5])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
20942024-11-13T05:13:10.614ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
20952024-11-13T05:13:10.614ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
20962024-11-13T05:13:10.614ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20972024-11-13T05:13:10.614ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20982024-11-13T05:13:10.614ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
20992024-11-13T05:13:10.615ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
21002024-11-13T05:13:10.615ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 325 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59814 req_id = 47cf78fd-cf35-465c-a0b3-16e41d86ca42 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
21012024-11-13T05:13:10.616ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33141
21022024-11-13T05:13:10.616ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21032024-11-13T05:13:10.665ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21042024-11-13T05:13:10.737ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 120201 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:33141 req_id = 261ab8dd-b83c-4d8f-a12e-e44108af36de response_code = 204 uri = /switch-ports
21052024-11-13T05:13:10.737ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33141 req_id = d5c2fa0f-d725-4006-8203-3586db82300d response_code = 200 uri = /network-bootstore-config
21062024-11-13T05:13:11.038ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47814
21072024-11-13T05:13:11.038ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47814 req_id = 3ca2a71f-cd24-4123-aba2-150154cde350 response_code = 200 uri = /vpc-routes
21082024-11-13T05:13:11.103ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
21092024-11-13T05:13:11.103ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:47814 req_id = 6ddbe05a-500e-4bbf-bb65-e8bb78103be2 response_code = 204 uri = /eip-gateways
21102024-11-13T05:13:11.122ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 208 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:47814 req_id = 5a8e747e-add4-4767-8d5d-d6ef6c15e596 response_code = 204 uri = /vpc-routes
21112024-11-13T05:13:11.228ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56709
21122024-11-13T05:13:11.228ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21132024-11-13T05:13:11.271ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21142024-11-13T05:13:11.347ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 119044 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:56709 req_id = 8b7e0891-38a5-41e5-bdac-3c77abf7ad94 response_code = 204 uri = /switch-ports
21152024-11-13T05:13:11.347ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54818
21162024-11-13T05:13:11.347ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51579
21172024-11-13T05:13:11.347ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54818 req_id = f8511cd2-42b9-4a63-ac16-e0e1af342758 response_code = 200 uri = /vpc-routes
21182024-11-13T05:13:11.347ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 262 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56709 req_id = 34470915-eacc-445b-b07a-27444b1c4871 response_code = 200 uri = /network-bootstore-config
21192024-11-13T05:13:11.347ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 247 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51579 req_id = 6f00b883-bc9b-46db-9b50-3d0d0020c6ed response_code = 200 uri = /v2p
21202024-11-13T05:13:11.390ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
21212024-11-13T05:13:11.390ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54818 req_id = 25f046b7-8b7c-4320-8e8e-ce31a9455b53 response_code = 204 uri = /eip-gateways
21222024-11-13T05:13:11.394ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46901
21232024-11-13T05:13:11.394ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46901 req_id = eb26d77a-3bb5-4115-9f05-b73e3ac6e8e2 response_code = 200 uri = /v2p
21242024-11-13T05:13:11.409ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 265 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54818 req_id = 1f1784ca-c9a0-4c4c-aba1-05233e58ddca response_code = 204 uri = /vpc-routes
21252024-11-13T05:13:11.411ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50812
21262024-11-13T05:13:11.412ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21272024-11-13T05:13:11.454ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21282024-11-13T05:13:11.514ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 102624 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:50812 req_id = ce24f297-07d8-4489-b086-f74f67edd103 response_code = 204 uri = /switch-ports
21292024-11-13T05:13:11.514ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50812 req_id = 318fd859-0860-4517-9d3e-bf4e8d5bd6e1 response_code = 200 uri = /network-bootstore-config
21302024-11-13T05:13:11.532ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64614
21312024-11-13T05:13:11.533ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 150e1804-b3f0-4f2c-82cf-fb2bbaab0984, kind: Service { id: 038e4c2b-2408-42f1-b025-c88f1167faaf }, name: Name("external-dns-038e4c2b-2408-42f1-b025-c88f1167faaf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 226, 83])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b98af94a-ceb1-41e9-82a5-9be8a66b4c85, kind: Service { id: eb4a98a4-3147-445f-9881-33f4dffeca66 }, name: Name("external-dns-eb4a98a4-3147-445f-9881-33f4dffeca66"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 145])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55f88ed5-95d7-43db-b06a-514b7adb4cdb, kind: Service { id: 7641195e-73b4-4972-ad51-4fcacb36a4be }, name: Name("nexus-7641195e-73b4-4972-ad51-4fcacb36a4be"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 204])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 984220cf-9c9a-412c-a465-3579565ca87a, kind: Service { id: e181617d-520a-429d-85ef-385f6dde38ea }, name: Name("nexus-e181617d-520a-429d-85ef-385f6dde38ea"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 83])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9df31eec-63bd-49fd-99e9-b3a05e952553, kind: Service { id: 6c4893ca-1b18-4910-8b18-504e5fcb8dc5 }, name: Name("nexus-6c4893ca-1b18-4910-8b18-504e5fcb8dc5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 5])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
21322024-11-13T05:13:11.533ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
21332024-11-13T05:13:11.533ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
21342024-11-13T05:13:11.533ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
21352024-11-13T05:13:11.533ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
21362024-11-13T05:13:11.533ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
21372024-11-13T05:13:11.533ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
21382024-11-13T05:13:11.533ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 400 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64614 req_id = bf376be8-4340-4e31-9485-4ca85fe39974 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
21392024-11-13T05:13:11.608ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57769
21402024-11-13T05:13:11.608ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 150e1804-b3f0-4f2c-82cf-fb2bbaab0984, kind: Service { id: 038e4c2b-2408-42f1-b025-c88f1167faaf }, name: Name("external-dns-038e4c2b-2408-42f1-b025-c88f1167faaf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 226, 83])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b98af94a-ceb1-41e9-82a5-9be8a66b4c85, kind: Service { id: eb4a98a4-3147-445f-9881-33f4dffeca66 }, name: Name("external-dns-eb4a98a4-3147-445f-9881-33f4dffeca66"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 145])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55f88ed5-95d7-43db-b06a-514b7adb4cdb, kind: Service { id: 7641195e-73b4-4972-ad51-4fcacb36a4be }, name: Name("nexus-7641195e-73b4-4972-ad51-4fcacb36a4be"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 204])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 984220cf-9c9a-412c-a465-3579565ca87a, kind: Service { id: e181617d-520a-429d-85ef-385f6dde38ea }, name: Name("nexus-e181617d-520a-429d-85ef-385f6dde38ea"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 83])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9df31eec-63bd-49fd-99e9-b3a05e952553, kind: Service { id: 6c4893ca-1b18-4910-8b18-504e5fcb8dc5 }, name: Name("nexus-6c4893ca-1b18-4910-8b18-504e5fcb8dc5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 5])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
21412024-11-13T05:13:11.608ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
21422024-11-13T05:13:11.608ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
21432024-11-13T05:13:11.608ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
21442024-11-13T05:13:11.608ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
21452024-11-13T05:13:11.608ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
21462024-11-13T05:13:11.608ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
21472024-11-13T05:13:11.608ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 237 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57769 req_id = d35cf0e7-0306-4b84-8b96-f332ed6fba5f response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
21482024-11-13T05:13:24.468ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:34365
21492024-11-13T05:13:24.468ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 47 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:34365 req_id = a836c104-142d-4267-b1de-f26b1d2857da response_code = 200 uri = /baseboard
21502024-11-13T05:13:40.129ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45375
21512024-11-13T05:13:40.130ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 42 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45375 req_id = 11a05f5f-8a6f-4dd5-83f0-0aa4ee755aa0 response_code = 200 uri = /vpc-routes
21522024-11-13T05:13:40.143ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
21532024-11-13T05:13:40.143ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45375 req_id = 2c183962-0b18-4900-9bd7-329a0202ee8e response_code = 204 uri = /eip-gateways
21542024-11-13T05:13:40.151ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 176 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45375 req_id = 4132187e-80d5-4684-83b6-179a9154f742 response_code = 204 uri = /vpc-routes
21552024-11-13T05:13:40.245ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37161
21562024-11-13T05:13:40.245ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37161 req_id = f7624be5-f2ab-46f6-bc5f-d0b51883b9f4 response_code = 200 uri = /v2p
21572024-11-13T05:13:40.322ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:65265
21582024-11-13T05:13:40.322ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21592024-11-13T05:13:40.363ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21602024-11-13T05:13:40.419ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 97615 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:65265 req_id = d2de8216-3b11-487b-be9f-af51eb38f53b response_code = 204 uri = /switch-ports
21612024-11-13T05:13:40.420ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:65265 req_id = bba8d10d-c2e1-49df-a1bd-f297e06d3f71 response_code = 200 uri = /network-bootstore-config
21622024-11-13T05:13:41.032ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64618
21632024-11-13T05:13:41.032ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64618 req_id = 06997287-d966-424e-98cb-a5d03c6b0a1d response_code = 200 uri = /vpc-routes
21642024-11-13T05:13:41.090ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
21652024-11-13T05:13:41.090ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64618 req_id = 0109b714-93a8-41a8-857d-b8d100bdeccf response_code = 204 uri = /eip-gateways
21662024-11-13T05:13:41.098ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 211 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64618 req_id = aa0cbe2b-aeda-4d01-885a-13e17c565d56 response_code = 204 uri = /vpc-routes
21672024-11-13T05:13:41.099ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54652
21682024-11-13T05:13:41.099ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21692024-11-13T05:13:41.144ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21702024-11-13T05:13:41.207ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 107697 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:54652 req_id = ee67b71a-03dd-4dfd-a96e-7d0589d4ce47 response_code = 204 uri = /switch-ports
21712024-11-13T05:13:41.207ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59634
21722024-11-13T05:13:41.207ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 159 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54652 req_id = 69991d5b-89a8-4ac7-92c2-7790164adeb0 response_code = 200 uri = /network-bootstore-config
21732024-11-13T05:13:41.207ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59634 req_id = e0b814d4-9890-4e54-98e9-5921614dba6e response_code = 200 uri = /vpc-routes
21742024-11-13T05:13:41.226ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45351
21752024-11-13T05:13:41.226ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 176 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45351 req_id = 52e84f0c-467f-455c-ad0f-428eb4f226f1 response_code = 200 uri = /v2p
21762024-11-13T05:13:41.242ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47411
21772024-11-13T05:13:41.242ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 122 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47411 req_id = 03855012-da9e-48e5-ade8-c642467e3aa6 response_code = 200 uri = /v2p
21782024-11-13T05:13:41.260ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45580
21792024-11-13T05:13:41.260ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21802024-11-13T05:13:41.302ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21812024-11-13T05:13:41.359ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 98808 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:45580 req_id = 94aed453-ae97-4f20-917e-9d812d269e0a response_code = 204 uri = /switch-ports
21822024-11-13T05:13:41.359ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
21832024-11-13T05:13:41.359ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 71 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:59634 req_id = 4e3eab2e-cebb-4cf5-837d-dd02c23477fe response_code = 204 uri = /eip-gateways
21842024-11-13T05:13:41.359ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45580 req_id = 37383214-48a6-4a96-83bd-4490ac8ad99d response_code = 200 uri = /network-bootstore-config
21852024-11-13T05:13:41.369ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 267 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:59634 req_id = dc60259c-8cfd-4575-a428-6efc73942937 response_code = 204 uri = /vpc-routes
21862024-11-13T05:13:54.500ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:48618
21872024-11-13T05:13:54.500ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 32 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:48618 req_id = 29969749-dc9d-4f49-b09d-32af55cee1dc response_code = 200 uri = /baseboard
21882024-11-13T05:14:10.243ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40346
21892024-11-13T05:14:10.243ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40346 req_id = bdab5a45-bdf7-49af-8801-ffde77c1ff1e response_code = 200 uri = /vpc-routes
21902024-11-13T05:14:10.308ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
21912024-11-13T05:14:10.308ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:40346 req_id = 3b1e8b79-a17b-445d-b929-5a60e5ebdc2a response_code = 204 uri = /eip-gateways
21922024-11-13T05:14:10.318ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 196 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:40346 req_id = f1b607fe-3016-4d33-95e4-2be463dd4c02 response_code = 204 uri = /vpc-routes
21932024-11-13T05:14:10.422ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:65237
21942024-11-13T05:14:10.422ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:65237 req_id = 3de793e6-c562-4211-b143-1d8aa941e6b4 response_code = 200 uri = /v2p
21952024-11-13T05:14:10.444ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50355
21962024-11-13T05:14:10.445ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21972024-11-13T05:14:10.483ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21982024-11-13T05:14:10.542ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 97150 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:50355 req_id = 1e594b7f-b30d-475b-88d7-dd52dc182c8a response_code = 204 uri = /switch-ports
21992024-11-13T05:14:10.542ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50355 req_id = 1f166f31-0f09-4d7e-92b4-d2674c21d9f2 response_code = 200 uri = /network-bootstore-config
22002024-11-13T05:14:11.127ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45871
22012024-11-13T05:14:11.127ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45871 req_id = e646dccc-690a-4723-828f-085e22050ad8 response_code = 200 uri = /vpc-routes
22022024-11-13T05:14:11.180ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
22032024-11-13T05:14:11.180ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:45871 req_id = a8190764-e80c-47fa-91d3-7b5a6404ef96 response_code = 204 uri = /eip-gateways
22042024-11-13T05:14:11.206ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64775
22052024-11-13T05:14:11.206ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64775 req_id = 0b458dc2-4dfb-4f44-b99c-fe542f0f0972 response_code = 200 uri = /vpc-routes
22062024-11-13T05:14:11.274ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57705
22072024-11-13T05:14:11.274ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57705 req_id = ded1b87b-828a-4085-88a3-840a221c0941 response_code = 200 uri = /v2p
22082024-11-13T05:14:11.274ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
22092024-11-13T05:14:11.274ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:64775 req_id = 94cba1cc-f651-4fa4-9388-083ba9637d58 response_code = 204 uri = /eip-gateways
22102024-11-13T05:14:11.277ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 234 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:45871 req_id = 003d29f4-9e13-4608-b953-af6e013e9fed response_code = 204 uri = /vpc-routes
22112024-11-13T05:14:11.288ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:64775 req_id = 3888a695-7d81-4ea1-bc67-01609569513b response_code = 204 uri = /vpc-routes
22122024-11-13T05:14:11.399ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49666
22132024-11-13T05:14:11.399ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
22142024-11-13T05:14:11.441ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
22152024-11-13T05:14:11.504ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 105248 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:49666 req_id = cf792bfe-5e0b-4b63-aa39-0a54f3d5cb8e response_code = 204 uri = /switch-ports
22162024-11-13T05:14:11.505ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49666 req_id = 2b7a7634-62ac-46c7-b4d8-39ffd3c5dda6 response_code = 200 uri = /network-bootstore-config
22172024-11-13T05:14:11.526ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42424
22182024-11-13T05:14:11.526ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42424 req_id = 26c260fb-d8af-4c86-98c1-409a35575478 response_code = 200 uri = /v2p
22192024-11-13T05:14:11.595ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40915
22202024-11-13T05:14:11.595ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
22212024-11-13T05:14:11.635ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
22222024-11-13T05:14:11.692ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 96454 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:40915 req_id = 14485035-194f-4c58-973d-ffc9304c8d6b response_code = 204 uri = /switch-ports
22232024-11-13T05:14:11.692ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40915 req_id = b0cd497c-e3cc-46b1-a1ec-1dbcc475335d response_code = 200 uri = /network-bootstore-config
22242024-11-13T05:14:24.534ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:45186
22252024-11-13T05:14:24.534ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 43 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:45186 req_id = 58483230-4922-417b-a9ed-e79cab552c2e response_code = 200 uri = /baseboard
22262024-11-13T05:14:40.156ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41842
22272024-11-13T05:14:40.156ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41842 req_id = e05174a7-c30b-4f19-9767-c5c4a1e0d35c response_code = 200 uri = /v2p
22282024-11-13T05:14:40.156ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55788
22292024-11-13T05:14:40.156ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 23 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55788 req_id = c68ad4eb-47a1-40ec-abd4-66802ed0f40d response_code = 200 uri = /vpc-routes
22302024-11-13T05:14:40.196ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
22312024-11-13T05:14:40.196ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:55788 req_id = 99af0faa-4002-4de0-be35-4100180da10b response_code = 204 uri = /eip-gateways
22322024-11-13T05:14:40.206ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 180 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:55788 req_id = c1dba79e-caee-407a-b36b-3244aca64284 response_code = 204 uri = /vpc-routes
22332024-11-13T05:14:40.222ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61126
22342024-11-13T05:14:40.222ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
22352024-11-13T05:14:40.268ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
22362024-11-13T05:14:40.334ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 112343 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:61126 req_id = f31d601c-c1d7-4a99-b473-8d6b26a3c149 response_code = 204 uri = /switch-ports
22372024-11-13T05:14:40.335ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61126 req_id = 312a8e65-3ac9-46a6-a68b-669f0cf5e2fc response_code = 200 uri = /network-bootstore-config
22382024-11-13T05:14:40.981ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48311
22392024-11-13T05:14:40.981ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48311 req_id = c5a22891-aaea-416c-853f-4a5cbf9fc19c response_code = 200 uri = /vpc-routes
22402024-11-13T05:14:41.015ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
22412024-11-13T05:14:41.015ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48311 req_id = 7c9799d3-cbcf-4d23-98df-51abe040838b response_code = 204 uri = /eip-gateways
22422024-11-13T05:14:41.026ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 375 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48311 req_id = a418b15f-83cc-4b64-a6d9-bb18cef381f3 response_code = 204 uri = /vpc-routes
22432024-11-13T05:14:41.109ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42571
22442024-11-13T05:14:41.110ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 140 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42571 req_id = 32435aba-83e7-4b17-afe5-496d40734f46 response_code = 200 uri = /v2p
22452024-11-13T05:14:41.124ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46671
22462024-11-13T05:14:41.124ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
22472024-11-13T05:14:41.164ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
22482024-11-13T05:14:41.217ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 93156 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:46671 req_id = fdb05ecd-f933-44bd-b285-70a0106a4859 response_code = 204 uri = /switch-ports
22492024-11-13T05:14:41.217ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53836
22502024-11-13T05:14:41.218ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53836 req_id = 05f30d7b-2255-4c66-8be3-03ce0d5b86f9 response_code = 200 uri = /vpc-routes
22512024-11-13T05:14:41.218ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 182 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46671 req_id = 66507333-dfd4-47e5-bd1d-aac0f8f472c6 response_code = 200 uri = /network-bootstore-config
22522024-11-13T05:14:41.237ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
22532024-11-13T05:14:41.237ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53836 req_id = e5bb4e99-3678-437c-ad92-efb5cca322dd response_code = 204 uri = /eip-gateways
22542024-11-13T05:14:41.246ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 264 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53836 req_id = 5d12cb92-4d5f-4d6b-b169-dfd4767f1c6c response_code = 204 uri = /vpc-routes
22552024-11-13T05:14:41.248ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57854
22562024-11-13T05:14:41.248ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57854 req_id = b90140db-ae6b-421e-bc8c-107e67e918ce response_code = 200 uri = /v2p
22572024-11-13T05:14:41.297ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33114
22582024-11-13T05:14:41.297ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
22592024-11-13T05:14:41.339ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
22602024-11-13T05:14:41.397ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 100388 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:33114 req_id = 77a40276-2c4a-4e42-9b24-fdadfdc18aea response_code = 204 uri = /switch-ports
22612024-11-13T05:14:41.398ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33114 req_id = 996e1037-187f-4c8a-80d3-6d05492d9f63 response_code = 200 uri = /network-bootstore-config
22622024-11-13T05:14:54.569ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:45298
22632024-11-13T05:14:54.569ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 30 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:45298 req_id = 69663ee7-0930-4658-a39a-2681e0370092 response_code = 200 uri = /baseboard
22642024-11-13T05:15:10.125ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37426
22652024-11-13T05:15:10.125ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37426 req_id = dd189f19-0ccf-4f15-851e-6f7b9e9dbced response_code = 200 uri = /vpc-routes
22662024-11-13T05:15:10.170ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
22672024-11-13T05:15:10.170ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 81 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:37426 req_id = c69fbe94-4a38-4cb9-8be4-dff46354a04e response_code = 204 uri = /eip-gateways
22682024-11-13T05:15:10.178ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 165 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:37426 req_id = 3615847e-8201-4dba-8357-1b928d3abf2c response_code = 204 uri = /vpc-routes
22692024-11-13T05:15:10.412ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50869
22702024-11-13T05:15:10.412ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50869 req_id = 1c847925-69b7-4c0e-a874-01d35b18791c response_code = 200 uri = /v2p
22712024-11-13T05:15:10.465ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34179
22722024-11-13T05:15:10.465ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
22732024-11-13T05:15:10.508ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
22742024-11-13T05:15:10.576ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 111522 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:34179 req_id = ad0ae4f2-3706-4cb0-acdc-a6059791fb2a response_code = 204 uri = /switch-ports
22752024-11-13T05:15:10.577ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34179 req_id = 0ccab279-e991-4e4b-adee-836cebdea939 response_code = 200 uri = /network-bootstore-config
22762024-11-13T05:15:11.060ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52891
22772024-11-13T05:15:11.060ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52891 req_id = 9f4fa9dd-9722-435d-b655-3a85855fa6fb response_code = 200 uri = /vpc-routes
22782024-11-13T05:15:11.136ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
22792024-11-13T05:15:11.136ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52891 req_id = 5fdbb118-b97c-42de-b775-28852836bd25 response_code = 204 uri = /eip-gateways
22802024-11-13T05:15:11.146ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 211 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52891 req_id = 392a1b04-982a-4765-8bf1-338d02b011f9 response_code = 204 uri = /vpc-routes
22812024-11-13T05:15:11.243ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38265
22822024-11-13T05:15:11.243ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
22832024-11-13T05:15:11.285ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
22842024-11-13T05:15:11.346ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 103222 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:38265 req_id = 8c776f5c-1223-430f-b966-1419867b84f3 response_code = 204 uri = /switch-ports
22852024-11-13T05:15:11.347ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47274
22862024-11-13T05:15:11.347ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59489
22872024-11-13T05:15:11.347ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55788
22882024-11-13T05:15:11.347ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 242 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55788 req_id = 6cac2413-5597-43d1-bad4-88360835e24c response_code = 200 uri = /v2p
22892024-11-13T05:15:11.347ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59489 req_id = a29a91e4-78b1-482c-acfb-7b5da7fd6a1a response_code = 200 uri = /vpc-routes
22902024-11-13T05:15:11.347ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 273 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47274 req_id = b0161b21-a255-42b5-802a-af7ba96430e9 response_code = 200 uri = /v2p
22912024-11-13T05:15:11.347ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 280 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38265 req_id = 317945c9-8a71-491c-964c-2596d0bfc7fa response_code = 200 uri = /network-bootstore-config
22922024-11-13T05:15:11.361ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
22932024-11-13T05:15:11.361ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 72 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:59489 req_id = caa51fcb-95bb-4f40-ac95-6fa2334ecc22 response_code = 204 uri = /eip-gateways
22942024-11-13T05:15:11.369ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 243 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:59489 req_id = 8e9fb72d-578b-4290-add1-c9e24e3f6324 response_code = 204 uri = /vpc-routes
22952024-11-13T05:15:11.423ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48766
22962024-11-13T05:15:11.423ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
22972024-11-13T05:15:11.462ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
22982024-11-13T05:15:11.522ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 99216 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:48766 req_id = 55d3b6ea-a24d-4dd2-8b95-cdac782b4940 response_code = 204 uri = /switch-ports
22992024-11-13T05:15:11.522ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48766 req_id = f216c678-01eb-48de-a20e-a4b3a2e3d3c8 response_code = 200 uri = /network-bootstore-config
23002024-11-13T05:15:24.603ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:46501
23012024-11-13T05:15:24.603ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 35 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:46501 req_id = ddab640b-9349-49bd-aad4-c4ebb091d802 response_code = 200 uri = /baseboard
23022024-11-13T05:15:29.372ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23032024-11-13T05:15:29.375ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23042024-11-13T05:15:29.376ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23052024-11-13T05:15:29.377ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23062024-11-13T05:15:29.377ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23072024-11-13T05:15:29.378ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23082024-11-13T05:15:29.379ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23092024-11-13T05:15:29.379ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23102024-11-13T05:15:29.380ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 498 n_new_samples = 6 n_overflow_samples = 4 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23112024-11-13T05:15:39.234ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23122024-11-13T05:15:39.373ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23132024-11-13T05:15:39.374ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23142024-11-13T05:15:39.375ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23152024-11-13T05:15:39.376ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23162024-11-13T05:15:39.376ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23172024-11-13T05:15:39.377ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23182024-11-13T05:15:39.378ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23192024-11-13T05:15:39.378ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23202024-11-13T05:15:39.379ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23212024-11-13T05:15:39.379ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23222024-11-13T05:15:39.380ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23232024-11-13T05:15:39.381ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23242024-11-13T05:15:39.381ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23252024-11-13T05:15:39.381ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23262024-11-13T05:15:39.382ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23272024-11-13T05:15:39.382ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23282024-11-13T05:15:39.383ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23292024-11-13T05:15:40.112ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51971
23302024-11-13T05:15:40.112ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 42 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51971 req_id = 1dc6bfa5-0eab-4340-9cfe-9cf9afea619d response_code = 200 uri = /vpc-routes
23312024-11-13T05:15:40.156ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
23322024-11-13T05:15:40.156ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51971 req_id = 58e53b93-8982-42d6-81eb-eca65a8fbb74 response_code = 204 uri = /eip-gateways
23332024-11-13T05:15:40.165ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 162 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51971 req_id = 63dd1306-195d-47ec-a45c-141b1cdab8b0 response_code = 204 uri = /vpc-routes
23342024-11-13T05:15:40.199ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53794
23352024-11-13T05:15:40.199ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53794 req_id = 274529b0-d3f9-4055-a288-fe2f1b4c223d response_code = 200 uri = /v2p
23362024-11-13T05:15:40.214ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56758
23372024-11-13T05:15:40.214ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
23382024-11-13T05:15:40.256ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
23392024-11-13T05:15:40.314ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 99944 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:56758 req_id = 87d1cc30-b7dd-4152-898f-f10493e23f6b response_code = 204 uri = /switch-ports
23402024-11-13T05:15:40.314ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56758 req_id = 6af38d79-38f5-4dac-958a-196604022a35 response_code = 200 uri = /network-bootstore-config
23412024-11-13T05:15:40.969ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48825
23422024-11-13T05:15:40.969ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48825 req_id = 199bc4a4-3c04-4239-a646-570638dfe5ac response_code = 200 uri = /vpc-routes
23432024-11-13T05:15:41.034ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
23442024-11-13T05:15:41.034ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48825 req_id = dba2b4e1-fa5b-4a6c-a207-4088c1083811 response_code = 204 uri = /eip-gateways
23452024-11-13T05:15:41.045ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 228 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48825 req_id = 087d3bec-c8e7-405c-921b-d46fd30e592f response_code = 204 uri = /vpc-routes
23462024-11-13T05:15:41.105ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55269
23472024-11-13T05:15:41.105ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55269 req_id = 487313a7-7f6b-4b07-a0a4-8af7f64e5577 response_code = 200 uri = /v2p
23482024-11-13T05:15:41.119ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63484
23492024-11-13T05:15:41.119ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
23502024-11-13T05:15:41.159ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
23512024-11-13T05:15:41.214ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 95395 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:63484 req_id = 1d4dce8d-3150-4634-a670-ba515fa88104 response_code = 204 uri = /switch-ports
23522024-11-13T05:15:41.214ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54717
23532024-11-13T05:15:41.214ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34216
23542024-11-13T05:15:41.214ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35716
23552024-11-13T05:15:41.215ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
23562024-11-13T05:15:41.215ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 294 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63484 req_id = ae14fcb8-05ef-435d-a14d-62d13c8c88a2 response_code = 200 uri = /network-bootstore-config
23572024-11-13T05:15:41.215ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54717 req_id = cd155c82-4a16-44ef-81da-2933f6504e6f response_code = 200 uri = /vpc-routes
23582024-11-13T05:15:41.219ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 4261 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34216 req_id = a69a6280-0dfa-4144-b44f-b75c86a8e11b response_code = 200 uri = /v2p
23592024-11-13T05:15:41.251ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
23602024-11-13T05:15:41.256ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
23612024-11-13T05:15:41.256ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54717 req_id = 1a5d2a0f-d3c4-4081-a851-66f59176d256 response_code = 204 uri = /eip-gateways
23622024-11-13T05:15:41.266ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 771 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54717 req_id = e27923bc-c124-4afa-aa43-472a6c6a7072 response_code = 204 uri = /vpc-routes
23632024-11-13T05:15:41.307ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 92097 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:35716 req_id = e695fac4-7a70-4916-94b7-34919365393e response_code = 204 uri = /switch-ports
23642024-11-13T05:15:41.307ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35716 req_id = 5ab18fc5-cafd-4348-9839-05494ecef09f response_code = 200 uri = /network-bootstore-config
23652024-11-13T05:15:49.239ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23662024-11-13T05:15:49.376ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23672024-11-13T05:15:49.377ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23682024-11-13T05:15:49.378ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23692024-11-13T05:15:49.379ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23702024-11-13T05:15:49.380ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23712024-11-13T05:15:49.381ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23722024-11-13T05:15:49.382ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23732024-11-13T05:15:49.383ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23742024-11-13T05:15:49.383ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23752024-11-13T05:15:49.384ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23762024-11-13T05:15:49.385ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23772024-11-13T05:15:49.385ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23782024-11-13T05:15:49.386ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23792024-11-13T05:15:49.386ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23802024-11-13T05:15:49.386ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23812024-11-13T05:15:49.387ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23822024-11-13T05:15:49.387ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23832024-11-13T05:15:54.636ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:62220
23842024-11-13T05:15:54.636ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 49 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:62220 req_id = 0bed496e-6d9b-4ea7-8cc7-9c2510ef15d3 response_code = 200 uri = /baseboard
23852024-11-13T05:15:59.244ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23862024-11-13T05:15:59.378ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23872024-11-13T05:15:59.379ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23882024-11-13T05:15:59.381ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23892024-11-13T05:15:59.381ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23902024-11-13T05:15:59.382ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23912024-11-13T05:15:59.384ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23922024-11-13T05:15:59.384ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23932024-11-13T05:15:59.385ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23942024-11-13T05:15:59.386ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23952024-11-13T05:15:59.386ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23962024-11-13T05:15:59.387ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23972024-11-13T05:15:59.388ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23982024-11-13T05:15:59.388ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
23992024-11-13T05:15:59.389ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24002024-11-13T05:15:59.389ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24012024-11-13T05:15:59.390ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24022024-11-13T05:15:59.390ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24032024-11-13T05:16:03.243ZINFOSledAgent (auto-cleanup-task): running automatic periodic zone bundle cleanup file = sled-agent/src/zone_bundle.rs:197
24042024-11-13T05:16:03.243ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
24052024-11-13T05:16:03.278ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 5944, tv_nsec: 788888937 } time_to_next_cleanup = 599.99999584s
24062024-11-13T05:16:09.249ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24072024-11-13T05:16:09.380ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24082024-11-13T05:16:09.381ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24092024-11-13T05:16:09.383ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24102024-11-13T05:16:09.383ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24112024-11-13T05:16:09.384ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24122024-11-13T05:16:09.386ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24132024-11-13T05:16:09.386ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24142024-11-13T05:16:09.387ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24152024-11-13T05:16:09.388ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24162024-11-13T05:16:09.388ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24172024-11-13T05:16:09.389ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24182024-11-13T05:16:09.389ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24192024-11-13T05:16:09.390ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24202024-11-13T05:16:09.390ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24212024-11-13T05:16:09.391ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24222024-11-13T05:16:09.391ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24232024-11-13T05:16:09.392ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24242024-11-13T05:16:10.181ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63691
24252024-11-13T05:16:10.181ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 43 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63691 req_id = 9b37f999-9268-4903-a016-a4d8ead81cd7 response_code = 200 uri = /vpc-routes
24262024-11-13T05:16:10.243ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
24272024-11-13T05:16:10.243ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:63691 req_id = 3ba20a47-560d-4947-b652-0c6acf3e2c71 response_code = 204 uri = /eip-gateways
24282024-11-13T05:16:10.252ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 190 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:63691 req_id = 8fcfa72c-5665-4330-989b-b38827930228 response_code = 204 uri = /vpc-routes
24292024-11-13T05:16:10.375ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61845
24302024-11-13T05:16:10.375ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61845 req_id = 07249c61-19ac-4f89-b00f-97e95f95fb7f response_code = 200 uri = /v2p
24312024-11-13T05:16:10.432ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40414
24322024-11-13T05:16:10.432ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
24332024-11-13T05:16:10.473ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
24342024-11-13T05:16:10.533ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 100868 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:40414 req_id = 91998812-a5e6-4ff9-ac6b-7732a67a8385 response_code = 204 uri = /switch-ports
24352024-11-13T05:16:10.534ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40414 req_id = 3301e94f-3b33-4496-af28-60f757c30fa8 response_code = 200 uri = /network-bootstore-config
24362024-11-13T05:16:11.167ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41982
24372024-11-13T05:16:11.167ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41982 req_id = 8f910b93-3af3-4a81-ba3c-7135725ad590 response_code = 200 uri = /vpc-routes
24382024-11-13T05:16:11.234ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55435
24392024-11-13T05:16:11.234ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55435 req_id = 6d874136-fedd-4e44-adcf-9f27ad77988c response_code = 200 uri = /vpc-routes
24402024-11-13T05:16:11.236ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
24412024-11-13T05:16:11.236ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:41982 req_id = eeadb048-3dd6-45f5-811b-f2ca59d122c7 response_code = 204 uri = /eip-gateways
24422024-11-13T05:16:11.246ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 208 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:41982 req_id = b0795f73-9f21-42cb-84fe-f06a3c6e1aae response_code = 204 uri = /vpc-routes
24432024-11-13T05:16:11.249ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
24442024-11-13T05:16:11.249ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 42 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55435 req_id = 7d115f24-f3b9-4060-9cf2-5144f1e5577f response_code = 204 uri = /eip-gateways
24452024-11-13T05:16:11.272ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35851
24462024-11-13T05:16:11.272ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 157 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35851 req_id = f33dda3f-719a-4151-9996-d22558b1476d response_code = 200 uri = /v2p
24472024-11-13T05:16:11.280ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 206 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55435 req_id = 3db1a7d0-65a1-4111-9bec-c3b3f15c36db response_code = 204 uri = /vpc-routes
24482024-11-13T05:16:11.339ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44477
24492024-11-13T05:16:11.339ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
24502024-11-13T05:16:11.382ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
24512024-11-13T05:16:11.443ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 103626 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:44477 req_id = 653bca6c-45db-4b33-bb74-2fa4990d3d24 response_code = 204 uri = /switch-ports
24522024-11-13T05:16:11.443ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44660
24532024-11-13T05:16:11.443ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44660 req_id = 7415baf7-5d5c-4f0b-b399-0cfb4fea6812 response_code = 200 uri = /v2p
24542024-11-13T05:16:11.443ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44477 req_id = 672a1aed-bb62-4deb-8e7c-e849d101ba60 response_code = 200 uri = /network-bootstore-config
24552024-11-13T05:16:11.446ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42620
24562024-11-13T05:16:11.446ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
24572024-11-13T05:16:11.488ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
24582024-11-13T05:16:11.546ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 99345 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:42620 req_id = cfa317d5-9923-451c-bccb-3dcb2ca85380 response_code = 204 uri = /switch-ports
24592024-11-13T05:16:11.546ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 141 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42620 req_id = 971135e9-75d8-4531-b117-a01255b73bde response_code = 200 uri = /network-bootstore-config
24602024-11-13T05:16:18.068ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
24612024-11-13T05:16:19.252ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24622024-11-13T05:16:19.382ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24632024-11-13T05:16:19.383ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24642024-11-13T05:16:19.385ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24652024-11-13T05:16:19.386ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24662024-11-13T05:16:19.386ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24672024-11-13T05:16:19.388ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24682024-11-13T05:16:19.389ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24692024-11-13T05:16:19.389ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24702024-11-13T05:16:19.390ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24712024-11-13T05:16:19.390ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24722024-11-13T05:16:19.391ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24732024-11-13T05:16:19.392ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24742024-11-13T05:16:19.392ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24752024-11-13T05:16:19.393ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24762024-11-13T05:16:19.393ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24772024-11-13T05:16:19.394ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24782024-11-13T05:16:19.394ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24792024-11-13T05:16:24.669ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:63162
24802024-11-13T05:16:24.670ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 81 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:63162 req_id = 84f325ed-2159-4d2a-86e3-24033f3fd7b0 response_code = 200 uri = /baseboard
24812024-11-13T05:16:29.256ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24822024-11-13T05:16:29.384ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24832024-11-13T05:16:29.385ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24842024-11-13T05:16:29.387ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24852024-11-13T05:16:29.388ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24862024-11-13T05:16:29.388ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24872024-11-13T05:16:29.390ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24882024-11-13T05:16:29.391ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24892024-11-13T05:16:29.391ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24902024-11-13T05:16:29.392ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24912024-11-13T05:16:29.392ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24922024-11-13T05:16:29.393ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24932024-11-13T05:16:29.394ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24942024-11-13T05:16:29.394ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24952024-11-13T05:16:29.395ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24962024-11-13T05:16:29.395ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24972024-11-13T05:16:29.395ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24982024-11-13T05:16:29.396ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
24992024-11-13T05:16:39.261ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25002024-11-13T05:16:39.386ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25012024-11-13T05:16:39.387ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25022024-11-13T05:16:39.389ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25032024-11-13T05:16:39.389ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25042024-11-13T05:16:39.390ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25052024-11-13T05:16:39.392ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25062024-11-13T05:16:39.392ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25072024-11-13T05:16:39.393ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25082024-11-13T05:16:39.394ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25092024-11-13T05:16:39.394ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25102024-11-13T05:16:39.395ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25112024-11-13T05:16:39.395ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25122024-11-13T05:16:39.396ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25132024-11-13T05:16:39.396ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25142024-11-13T05:16:39.397ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25152024-11-13T05:16:39.397ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25162024-11-13T05:16:39.398ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25172024-11-13T05:16:40.090ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42001
25182024-11-13T05:16:40.090ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 41 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42001 req_id = 4b48366e-3ef4-442d-879a-459010a47ff1 response_code = 200 uri = /vpc-routes
25192024-11-13T05:16:40.105ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
25202024-11-13T05:16:40.105ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42001 req_id = 20009cb4-8936-4cc1-aa99-80c07f80f863 response_code = 204 uri = /eip-gateways
25212024-11-13T05:16:40.116ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 181 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42001 req_id = 4be07612-275d-4a45-be4d-b5a17ba5851e response_code = 204 uri = /vpc-routes
25222024-11-13T05:16:40.186ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33143
25232024-11-13T05:16:40.186ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 175 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33143 req_id = 44a6798a-f565-44c1-87dc-034fc90f45c5 response_code = 200 uri = /v2p
25242024-11-13T05:16:40.188ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61832
25252024-11-13T05:16:40.188ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
25262024-11-13T05:16:40.228ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
25272024-11-13T05:16:40.285ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 97255 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:61832 req_id = 2f594135-2d29-4155-a9dc-090a848dd556 response_code = 204 uri = /switch-ports
25282024-11-13T05:16:40.285ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61832 req_id = 4dca7e5b-bfe3-46d9-b70f-c6a8db90bc27 response_code = 200 uri = /network-bootstore-config
25292024-11-13T05:16:41.028ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49192
25302024-11-13T05:16:41.028ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49192 req_id = acf10deb-79aa-47b6-8545-917b9037f775 response_code = 200 uri = /vpc-routes
25312024-11-13T05:16:41.084ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
25322024-11-13T05:16:41.084ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:49192 req_id = 903e29e3-b6d5-4038-a848-a5f415db9cd3 response_code = 204 uri = /eip-gateways
25332024-11-13T05:16:41.129ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41362
25342024-11-13T05:16:41.129ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 186 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41362 req_id = 4d554251-1ea7-44ac-ada4-2a9ed9ad1312 response_code = 200 uri = /v2p
25352024-11-13T05:16:41.140ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 224 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:49192 req_id = c1913e84-af52-4373-bb89-2acd25554511 response_code = 204 uri = /vpc-routes
25362024-11-13T05:16:41.150ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48352
25372024-11-13T05:16:41.150ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
25382024-11-13T05:16:41.210ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
25392024-11-13T05:16:41.301ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 150787 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:48352 req_id = 1e261bd5-b960-4945-ab8a-547ac8326f94 response_code = 204 uri = /switch-ports
25402024-11-13T05:16:41.301ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65020
25412024-11-13T05:16:41.302ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65020 req_id = f1cc11a5-6233-454d-be82-55acb1b91e16 response_code = 200 uri = /vpc-routes
25422024-11-13T05:16:41.302ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 214 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48352 req_id = e9b64d42-9881-4575-8fd5-d2aab009c88b response_code = 200 uri = /network-bootstore-config
25432024-11-13T05:16:41.326ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37397
25442024-11-13T05:16:41.326ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
25452024-11-13T05:16:41.379ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
25462024-11-13T05:16:41.463ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 137640 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:37397 req_id = 77085d87-7a8a-4e46-8d18-8b6625ec5f8e response_code = 204 uri = /switch-ports
25472024-11-13T05:16:41.464ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34641
25482024-11-13T05:16:41.464ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
25492024-11-13T05:16:41.464ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:65020 req_id = 0766bd63-7d5f-4081-ae8c-0914f2ed6233 response_code = 204 uri = /eip-gateways
25502024-11-13T05:16:41.464ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 247 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37397 req_id = 80ca2de8-91a9-447e-b26c-5b8bf67911f5 response_code = 200 uri = /network-bootstore-config
25512024-11-13T05:16:41.464ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 236 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34641 req_id = 1bf598b3-6808-4c10-93eb-076aa51d6d8c response_code = 200 uri = /v2p
25522024-11-13T05:16:41.473ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:65020 req_id = d4ad10a3-733b-4164-9151-38344c8299d6 response_code = 204 uri = /vpc-routes
25532024-11-13T05:16:49.266ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25542024-11-13T05:16:49.389ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25552024-11-13T05:16:49.389ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25562024-11-13T05:16:49.390ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25572024-11-13T05:16:49.391ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25582024-11-13T05:16:49.392ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25592024-11-13T05:16:49.393ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25602024-11-13T05:16:49.394ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25612024-11-13T05:16:49.395ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25622024-11-13T05:16:49.396ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25632024-11-13T05:16:49.397ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25642024-11-13T05:16:49.397ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25652024-11-13T05:16:49.398ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25662024-11-13T05:16:49.398ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25672024-11-13T05:16:49.399ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25682024-11-13T05:16:49.399ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25692024-11-13T05:16:49.400ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25702024-11-13T05:16:49.400ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25712024-11-13T05:16:54.701ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:35581
25722024-11-13T05:16:54.701ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 40 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:35581 req_id = 5222e56b-00a2-4b87-87d5-8d4e27262c12 response_code = 200 uri = /baseboard
25732024-11-13T05:16:59.269ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25742024-11-13T05:16:59.391ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25752024-11-13T05:16:59.392ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25762024-11-13T05:16:59.392ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25772024-11-13T05:16:59.393ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25782024-11-13T05:16:59.394ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25792024-11-13T05:16:59.395ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25802024-11-13T05:16:59.396ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25812024-11-13T05:16:59.397ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25822024-11-13T05:16:59.398ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25832024-11-13T05:16:59.399ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25842024-11-13T05:16:59.400ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25852024-11-13T05:16:59.400ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25862024-11-13T05:16:59.400ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25872024-11-13T05:16:59.401ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25882024-11-13T05:16:59.401ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25892024-11-13T05:16:59.402ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25902024-11-13T05:16:59.402ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25912024-11-13T05:17:09.273ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25922024-11-13T05:17:09.394ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25932024-11-13T05:17:09.394ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25942024-11-13T05:17:09.395ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25952024-11-13T05:17:09.396ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25962024-11-13T05:17:09.396ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25972024-11-13T05:17:09.397ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25982024-11-13T05:17:09.398ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
25992024-11-13T05:17:09.398ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26002024-11-13T05:17:09.401ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26012024-11-13T05:17:09.401ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26022024-11-13T05:17:09.402ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26032024-11-13T05:17:09.402ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26042024-11-13T05:17:09.403ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26052024-11-13T05:17:09.403ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26062024-11-13T05:17:09.404ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26072024-11-13T05:17:09.404ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26082024-11-13T05:17:09.405ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26092024-11-13T05:17:10.341ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55720
26102024-11-13T05:17:10.341ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 52 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55720 req_id = fbe1e7e4-0b7b-4389-ad41-06948be96e75 response_code = 200 uri = /vpc-routes
26112024-11-13T05:17:10.387ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
26122024-11-13T05:17:10.387ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 77 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:55720 req_id = 3ccc2aa0-b998-499e-ba63-6fb1785394e1 response_code = 204 uri = /eip-gateways
26132024-11-13T05:17:10.395ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 158 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:55720 req_id = d08ee24b-f4c6-49f8-a728-bde2fa99b700 response_code = 204 uri = /vpc-routes
26142024-11-13T05:17:10.491ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34521
26152024-11-13T05:17:10.492ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34521 req_id = 28a495b6-b9f3-4047-aa35-11eb7d624bd1 response_code = 200 uri = /v2p
26162024-11-13T05:17:10.562ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:32889
26172024-11-13T05:17:10.562ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
26182024-11-13T05:17:10.606ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
26192024-11-13T05:17:10.666ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 103788 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:32889 req_id = 64e18949-1b83-435c-b917-66d27fe1688f response_code = 204 uri = /switch-ports
26202024-11-13T05:17:10.672ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:32889 req_id = 0e6759d3-b3c1-4be6-8138-bec14422de5b response_code = 200 uri = /network-bootstore-config
26212024-11-13T05:17:11.112ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45065
26222024-11-13T05:17:11.112ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45065 req_id = eb963029-3dfa-41f9-aeab-b7fbc15c5faa response_code = 200 uri = /vpc-routes
26232024-11-13T05:17:11.176ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
26242024-11-13T05:17:11.176ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:45065 req_id = b5cc62b5-dc86-43a0-838c-15a176cdc553 response_code = 204 uri = /eip-gateways
26252024-11-13T05:17:11.227ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56427
26262024-11-13T05:17:11.227ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56427 req_id = 87bb7742-9882-4caa-b0bf-110348b1aea8 response_code = 200 uri = /v2p
26272024-11-13T05:17:11.236ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 171 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:45065 req_id = d467ed6c-f458-4f34-b9d6-8092de5c7c74 response_code = 204 uri = /vpc-routes
26282024-11-13T05:17:11.276ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61072
26292024-11-13T05:17:11.276ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 32 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61072 req_id = 51084814-864a-4c7b-a40b-c43faeb53793 response_code = 200 uri = /vpc-routes
26302024-11-13T05:17:11.291ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
26312024-11-13T05:17:11.291ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:61072 req_id = cc00650c-62b5-4970-9f30-53e5cb134f6f response_code = 204 uri = /eip-gateways
26322024-11-13T05:17:11.299ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51665
26332024-11-13T05:17:11.299ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
26342024-11-13T05:17:11.337ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
26352024-11-13T05:17:11.395ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 96023 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:51665 req_id = 1434f341-c10e-4258-8c2c-4fdb3d702a7e response_code = 204 uri = /switch-ports
26362024-11-13T05:17:11.395ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33147
26372024-11-13T05:17:11.395ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 333 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:61072 req_id = 5111227a-8860-47a1-afa3-cfb789a56dfe response_code = 204 uri = /vpc-routes
26382024-11-13T05:17:11.395ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 265 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51665 req_id = 5dcc7b51-4cdb-4deb-98b1-eab74b33b075 response_code = 200 uri = /network-bootstore-config
26392024-11-13T05:17:11.395ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
26402024-11-13T05:17:11.403ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60621
26412024-11-13T05:17:11.403ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 241 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60621 req_id = f278df8a-b06d-4d24-9df1-931072be8023 response_code = 200 uri = /v2p
26422024-11-13T05:17:11.436ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
26432024-11-13T05:17:11.498ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 102533 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:33147 req_id = 6bf9a774-745c-4b59-941a-1635d0e51e37 response_code = 204 uri = /switch-ports
26442024-11-13T05:17:11.498ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33147 req_id = 7fb8a49d-132f-41d3-a6e5-165c6a1e1b7b response_code = 200 uri = /network-bootstore-config
26452024-11-13T05:17:19.277ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26462024-11-13T05:17:19.395ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26472024-11-13T05:17:19.396ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26482024-11-13T05:17:19.398ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26492024-11-13T05:17:19.399ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26502024-11-13T05:17:19.399ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26512024-11-13T05:17:19.400ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26522024-11-13T05:17:19.401ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26532024-11-13T05:17:19.401ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26542024-11-13T05:17:19.402ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26552024-11-13T05:17:19.404ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26562024-11-13T05:17:19.404ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26572024-11-13T05:17:19.405ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26582024-11-13T05:17:19.405ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26592024-11-13T05:17:19.406ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26602024-11-13T05:17:19.406ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26612024-11-13T05:17:19.407ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26622024-11-13T05:17:19.407ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26632024-11-13T05:17:24.735ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:48267
26642024-11-13T05:17:24.736ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 54 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:48267 req_id = e9ad1391-410a-4cf5-830f-5005ee904d68 response_code = 200 uri = /baseboard
26652024-11-13T05:17:29.282ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26662024-11-13T05:17:29.398ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26672024-11-13T05:17:29.399ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26682024-11-13T05:17:29.400ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26692024-11-13T05:17:29.400ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26702024-11-13T05:17:29.402ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26712024-11-13T05:17:29.403ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26722024-11-13T05:17:29.403ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26732024-11-13T05:17:29.404ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26742024-11-13T05:17:29.405ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26752024-11-13T05:17:29.406ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26762024-11-13T05:17:29.407ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26772024-11-13T05:17:29.407ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26782024-11-13T05:17:29.408ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26792024-11-13T05:17:29.408ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26802024-11-13T05:17:29.409ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26812024-11-13T05:17:29.409ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26822024-11-13T05:17:29.410ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26832024-11-13T05:17:39.285ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26842024-11-13T05:17:39.401ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26852024-11-13T05:17:39.401ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26862024-11-13T05:17:39.402ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26872024-11-13T05:17:39.403ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26882024-11-13T05:17:39.404ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26892024-11-13T05:17:39.404ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26902024-11-13T05:17:39.406ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26912024-11-13T05:17:39.407ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26922024-11-13T05:17:39.407ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26932024-11-13T05:17:39.408ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26942024-11-13T05:17:39.409ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26952024-11-13T05:17:39.409ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26962024-11-13T05:17:39.410ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26972024-11-13T05:17:39.410ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26982024-11-13T05:17:39.411ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
26992024-11-13T05:17:39.411ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27002024-11-13T05:17:39.412ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27012024-11-13T05:17:40.050ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34104
27022024-11-13T05:17:40.050ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34104 req_id = 6e139e5b-815c-458e-b29a-144622e6d610 response_code = 200 uri = /vpc-routes
27032024-11-13T05:17:40.078ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
27042024-11-13T05:17:40.078ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 81 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:34104 req_id = edca7060-49de-4675-8d89-c52dd1f87fff response_code = 204 uri = /eip-gateways
27052024-11-13T05:17:40.181ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33285
27062024-11-13T05:17:40.181ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33285 req_id = 5bcb0229-7867-4488-b200-fc6fcfa787e3 response_code = 200 uri = /v2p
27072024-11-13T05:17:40.187ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 180 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:34104 req_id = e4193c00-405d-4e1c-9743-50f1cc79dff5 response_code = 204 uri = /vpc-routes
27082024-11-13T05:17:40.264ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59294
27092024-11-13T05:17:40.264ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
27102024-11-13T05:17:40.306ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
27112024-11-13T05:17:40.366ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 102136 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:59294 req_id = 12d12e2c-3f82-41db-8c39-abcc4ab15c32 response_code = 204 uri = /switch-ports
27122024-11-13T05:17:40.366ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59294 req_id = 6e413ed1-0615-443b-9683-8e7ff37ba1df response_code = 200 uri = /network-bootstore-config
27132024-11-13T05:17:40.979ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46149
27142024-11-13T05:17:40.980ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46149 req_id = c3b59f6f-327d-4ba0-a09f-e2ba86bac215 response_code = 200 uri = /vpc-routes
27152024-11-13T05:17:41.008ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
27162024-11-13T05:17:41.008ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:46149 req_id = 0bfbc780-2cbb-403e-adfa-cbaf1bc5a6ff response_code = 204 uri = /eip-gateways
27172024-11-13T05:17:41.032ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 287 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:46149 req_id = 6c22baf9-6add-44e5-836e-c1a032ca9e79 response_code = 204 uri = /vpc-routes
27182024-11-13T05:17:41.122ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59296
27192024-11-13T05:17:41.122ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
27202024-11-13T05:17:41.178ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
27212024-11-13T05:17:41.269ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 147122 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:59296 req_id = c1c0dcf1-1649-4a1e-85b2-329d004326d0 response_code = 204 uri = /switch-ports
27222024-11-13T05:17:41.269ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36288
27232024-11-13T05:17:41.269ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49368
27242024-11-13T05:17:41.270ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 266 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59296 req_id = 4bc1dff4-7ec0-4b7a-9428-b8c38aef0b21 response_code = 200 uri = /network-bootstore-config
27252024-11-13T05:17:41.270ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 243 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49368 req_id = de5a52eb-b510-4b02-8e43-029743d56a5b response_code = 200 uri = /v2p
27262024-11-13T05:17:41.270ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 203 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36288 req_id = c2426111-f0f1-4674-985f-f2a6b11f3211 response_code = 200 uri = /vpc-routes
27272024-11-13T05:17:41.323ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33920
27282024-11-13T05:17:41.323ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
27292024-11-13T05:17:41.371ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
27302024-11-13T05:17:41.430ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 107437 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:33920 req_id = bfff737d-0ed7-43ed-a70c-bbf53bf46330 response_code = 204 uri = /switch-ports
27312024-11-13T05:17:41.430ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64852
27322024-11-13T05:17:41.431ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
27332024-11-13T05:17:41.431ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36288 req_id = 0ae79186-0119-4762-9931-f93713014944 response_code = 204 uri = /eip-gateways
27342024-11-13T05:17:41.431ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 280 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33920 req_id = b55dd993-a9f9-45ea-819f-ec608a69abec response_code = 200 uri = /network-bootstore-config
27352024-11-13T05:17:41.431ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 242 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64852 req_id = 4be56865-ef49-4e6b-bba3-bd92c6bcbcf2 response_code = 200 uri = /v2p
27362024-11-13T05:17:41.442ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 315 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36288 req_id = f4d49dc8-2eda-44e8-a2c9-7f070d0b00a1 response_code = 204 uri = /vpc-routes
27372024-11-13T05:17:49.289ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27382024-11-13T05:17:49.403ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27392024-11-13T05:17:49.404ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27402024-11-13T05:17:49.404ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27412024-11-13T05:17:49.405ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27422024-11-13T05:17:49.406ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27432024-11-13T05:17:49.406ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27442024-11-13T05:17:49.408ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27452024-11-13T05:17:49.409ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27462024-11-13T05:17:49.409ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27472024-11-13T05:17:49.410ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27482024-11-13T05:17:49.410ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27492024-11-13T05:17:49.411ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27502024-11-13T05:17:49.411ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27512024-11-13T05:17:49.412ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27522024-11-13T05:17:49.412ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27532024-11-13T05:17:49.413ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27542024-11-13T05:17:49.413ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27552024-11-13T05:17:54.769ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:61827
27562024-11-13T05:17:54.769ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 78 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:61827 req_id = cce0a178-1639-4473-9068-193950f9680c response_code = 200 uri = /baseboard
27572024-11-13T05:17:59.293ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27582024-11-13T05:17:59.405ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27592024-11-13T05:17:59.406ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27602024-11-13T05:17:59.407ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27612024-11-13T05:17:59.408ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27622024-11-13T05:17:59.408ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27632024-11-13T05:17:59.409ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27642024-11-13T05:17:59.410ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27652024-11-13T05:17:59.411ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27662024-11-13T05:17:59.412ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27672024-11-13T05:17:59.412ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27682024-11-13T05:17:59.413ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27692024-11-13T05:17:59.414ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27702024-11-13T05:17:59.414ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27712024-11-13T05:17:59.415ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27722024-11-13T05:17:59.415ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27732024-11-13T05:17:59.416ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27742024-11-13T05:17:59.416ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27752024-11-13T05:18:09.296ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27762024-11-13T05:18:09.407ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27772024-11-13T05:18:09.408ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27782024-11-13T05:18:09.409ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27792024-11-13T05:18:09.409ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27802024-11-13T05:18:09.410ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27812024-11-13T05:18:09.411ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27822024-11-13T05:18:09.411ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27832024-11-13T05:18:09.413ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27842024-11-13T05:18:09.414ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27852024-11-13T05:18:09.414ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27862024-11-13T05:18:09.415ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27872024-11-13T05:18:09.415ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27882024-11-13T05:18:09.416ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27892024-11-13T05:18:09.416ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27902024-11-13T05:18:09.417ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27912024-11-13T05:18:09.417ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27922024-11-13T05:18:09.418ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
27932024-11-13T05:18:10.442ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45177
27942024-11-13T05:18:10.442ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45177 req_id = 18fd8327-1419-4847-be50-f27c1491a903 response_code = 200 uri = /v2p
27952024-11-13T05:18:10.475ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42233
27962024-11-13T05:18:10.475ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 36 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42233 req_id = daf23b49-14be-41c2-a652-480388bb3f93 response_code = 200 uri = /vpc-routes
27972024-11-13T05:18:10.489ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
27982024-11-13T05:18:10.489ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42233 req_id = 14de0de3-46a5-4a29-8f3d-2dde98b20d95 response_code = 204 uri = /eip-gateways
27992024-11-13T05:18:10.494ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51191
28002024-11-13T05:18:10.494ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 150e1804-b3f0-4f2c-82cf-fb2bbaab0984, kind: Service { id: 038e4c2b-2408-42f1-b025-c88f1167faaf }, name: Name("external-dns-038e4c2b-2408-42f1-b025-c88f1167faaf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 226, 83])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b98af94a-ceb1-41e9-82a5-9be8a66b4c85, kind: Service { id: eb4a98a4-3147-445f-9881-33f4dffeca66 }, name: Name("external-dns-eb4a98a4-3147-445f-9881-33f4dffeca66"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 145])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55f88ed5-95d7-43db-b06a-514b7adb4cdb, kind: Service { id: 7641195e-73b4-4972-ad51-4fcacb36a4be }, name: Name("nexus-7641195e-73b4-4972-ad51-4fcacb36a4be"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 204])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 984220cf-9c9a-412c-a465-3579565ca87a, kind: Service { id: e181617d-520a-429d-85ef-385f6dde38ea }, name: Name("nexus-e181617d-520a-429d-85ef-385f6dde38ea"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 83])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9df31eec-63bd-49fd-99e9-b3a05e952553, kind: Service { id: 6c4893ca-1b18-4910-8b18-504e5fcb8dc5 }, name: Name("nexus-6c4893ca-1b18-4910-8b18-504e5fcb8dc5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 5])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
28012024-11-13T05:18:10.495ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
28022024-11-13T05:18:10.495ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
28032024-11-13T05:18:10.495ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
28042024-11-13T05:18:10.495ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
28052024-11-13T05:18:10.495ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
28062024-11-13T05:18:10.495ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
28072024-11-13T05:18:10.495ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 244 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51191 req_id = 4f0fe75f-bf47-4027-9e89-4f921e416272 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
28082024-11-13T05:18:10.502ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42233 req_id = 6d8f5711-e8cc-4854-b990-b455f46f2c51 response_code = 204 uri = /vpc-routes
28092024-11-13T05:18:10.604ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58586
28102024-11-13T05:18:10.604ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
28112024-11-13T05:18:10.604ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
28122024-11-13T05:18:10.604ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/omicron-zones.json file = common/src/ledger.rs:177
28132024-11-13T05:18:10.604ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/omicron-zones.json file = common/src/ledger.rs:177
28142024-11-13T05:18:10.618ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28152024-11-13T05:18:10.618ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8511034a-a894-457f-9cbc-98e6461f9f5a
28162024-11-13T05:18:10.641ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28172024-11-13T05:18:10.641ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c867bbf7-921c-4db1-887a-4002bcc6b446
28182024-11-13T05:18:10.668ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28192024-11-13T05:18:10.668ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_204f40c4-4e09-4726-a368-a3da2ac1b831
28202024-11-13T05:18:10.687ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49647
28212024-11-13T05:18:10.688ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
28222024-11-13T05:18:10.693ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28232024-11-13T05:18:10.693ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8e821c87-2d05-4e93-b160-49a645a177c5
28242024-11-13T05:18:10.721ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28252024-11-13T05:18:10.721ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_e770fce0-3dd5-4678-9037-68ea781dbb30
28262024-11-13T05:18:10.732ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
28272024-11-13T05:18:10.751ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28282024-11-13T05:18:10.751ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c2ce9d45-3d62-4234-a9db-c1d5c8884adc
28292024-11-13T05:18:10.780ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28302024-11-13T05:18:10.780ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c9fdd7dd-67c5-4597-a857-03bc9769dc87
28312024-11-13T05:18:10.803ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 115097 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:49647 req_id = 715d223c-db22-46c2-a287-85ae60c1e5b3 response_code = 204 uri = /switch-ports
28322024-11-13T05:18:10.803ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 163 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49647 req_id = 9d5cf88a-f328-48c6-93f5-5e443b98ae70 response_code = 200 uri = /network-bootstore-config
28332024-11-13T05:18:10.809ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28342024-11-13T05:18:10.809ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_27d5a04d-ccb1-4219-851b-abc94499936f
28352024-11-13T05:18:10.848ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28362024-11-13T05:18:10.848ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c7851ba0-6692-4564-8a56-a39ae4767989
28372024-11-13T05:18:10.868ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 264204 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58586 req_id = 4f9003ad-0297-4098-8fd8-bbbed563d780 response_code = 200 uri = /inventory
28382024-11-13T05:18:11.162ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52692
28392024-11-13T05:18:11.162ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52692 req_id = 3df2ca89-ba4c-4fda-a1a1-c33ff74080a4 response_code = 200 uri = /vpc-routes
28402024-11-13T05:18:11.216ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
28412024-11-13T05:18:11.216ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52692 req_id = a9618c0c-8078-4f70-a3d5-240672469af3 response_code = 204 uri = /eip-gateways
28422024-11-13T05:18:11.271ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48392
28432024-11-13T05:18:11.271ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48392 req_id = b5057c05-0892-49a6-8a8e-6d9a2461518c response_code = 200 uri = /vpc-routes
28442024-11-13T05:18:11.316ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
28452024-11-13T05:18:11.316ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48392 req_id = bc01d5e8-37ab-432d-b577-b0050410cbe9 response_code = 204 uri = /eip-gateways
28462024-11-13T05:18:11.437ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61657
28472024-11-13T05:18:11.437ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 183 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61657 req_id = ba7f2d4e-6132-4a86-9c2c-de5d0e06e0a9 response_code = 200 uri = /v2p
28482024-11-13T05:18:11.448ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 218 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48392 req_id = 5166a349-2263-4d79-9a46-96786787cba0 response_code = 204 uri = /vpc-routes
28492024-11-13T05:18:11.492ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37893
28502024-11-13T05:18:11.492ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
28512024-11-13T05:18:11.492ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
28522024-11-13T05:18:11.492ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/omicron-zones.json file = common/src/ledger.rs:177
28532024-11-13T05:18:11.492ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/omicron-zones.json file = common/src/ledger.rs:177
28542024-11-13T05:18:11.504ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 324 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52692 req_id = d3bb0e66-a1bd-45c2-ac36-10927209db80 response_code = 204 uri = /vpc-routes
28552024-11-13T05:18:11.507ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28562024-11-13T05:18:11.507ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8511034a-a894-457f-9cbc-98e6461f9f5a
28572024-11-13T05:18:11.537ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28582024-11-13T05:18:11.537ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c867bbf7-921c-4db1-887a-4002bcc6b446
28592024-11-13T05:18:11.560ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28602024-11-13T05:18:11.560ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_204f40c4-4e09-4726-a368-a3da2ac1b831
28612024-11-13T05:18:11.578ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54531
28622024-11-13T05:18:11.579ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54531 req_id = e2b79802-5970-4e6a-90b2-ad2d5b26bc7a response_code = 200 uri = /v2p
28632024-11-13T05:18:11.580ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28642024-11-13T05:18:11.580ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8e821c87-2d05-4e93-b160-49a645a177c5
28652024-11-13T05:18:11.597ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44717
28662024-11-13T05:18:11.597ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
28672024-11-13T05:18:11.597ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
28682024-11-13T05:18:11.597ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/config/omicron-zones.json file = common/src/ledger.rs:177
28692024-11-13T05:18:11.597ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/config/omicron-zones.json file = common/src/ledger.rs:177
28702024-11-13T05:18:11.602ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28712024-11-13T05:18:11.602ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_e770fce0-3dd5-4678-9037-68ea781dbb30
28722024-11-13T05:18:11.612ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49850
28732024-11-13T05:18:11.612ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
28742024-11-13T05:18:11.618ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8511034a-a894-457f-9cbc-98e6461f9f5a (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28752024-11-13T05:18:11.618ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8511034a-a894-457f-9cbc-98e6461f9f5a
28762024-11-13T05:18:11.647ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61667
28772024-11-13T05:18:11.652ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c867bbf7-921c-4db1-887a-4002bcc6b446 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28782024-11-13T05:18:11.653ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c867bbf7-921c-4db1-887a-4002bcc6b446
28792024-11-13T05:18:11.659ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
28802024-11-13T05:18:11.670ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39860
28812024-11-13T05:18:11.670ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 150e1804-b3f0-4f2c-82cf-fb2bbaab0984, kind: Service { id: 038e4c2b-2408-42f1-b025-c88f1167faaf }, name: Name("external-dns-038e4c2b-2408-42f1-b025-c88f1167faaf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 226, 83])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b98af94a-ceb1-41e9-82a5-9be8a66b4c85, kind: Service { id: eb4a98a4-3147-445f-9881-33f4dffeca66 }, name: Name("external-dns-eb4a98a4-3147-445f-9881-33f4dffeca66"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 145])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55f88ed5-95d7-43db-b06a-514b7adb4cdb, kind: Service { id: 7641195e-73b4-4972-ad51-4fcacb36a4be }, name: Name("nexus-7641195e-73b4-4972-ad51-4fcacb36a4be"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 204])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 984220cf-9c9a-412c-a465-3579565ca87a, kind: Service { id: e181617d-520a-429d-85ef-385f6dde38ea }, name: Name("nexus-e181617d-520a-429d-85ef-385f6dde38ea"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 83])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9df31eec-63bd-49fd-99e9-b3a05e952553, kind: Service { id: 6c4893ca-1b18-4910-8b18-504e5fcb8dc5 }, name: Name("nexus-6c4893ca-1b18-4910-8b18-504e5fcb8dc5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 5])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
28822024-11-13T05:18:11.670ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
28832024-11-13T05:18:11.670ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
28842024-11-13T05:18:11.670ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
28852024-11-13T05:18:11.670ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
28862024-11-13T05:18:11.670ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
28872024-11-13T05:18:11.670ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
28882024-11-13T05:18:11.670ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 373 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39860 req_id = d14a65cc-963c-4340-ace1-6aee38fadb0a response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
28892024-11-13T05:18:11.671ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28902024-11-13T05:18:11.671ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c2ce9d45-3d62-4234-a9db-c1d5c8884adc
28912024-11-13T05:18:11.691ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 204f40c4-4e09-4726-a368-a3da2ac1b831 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28922024-11-13T05:18:11.691ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_204f40c4-4e09-4726-a368-a3da2ac1b831
28932024-11-13T05:18:11.705ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63415
28942024-11-13T05:18:11.705ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:731 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 150e1804-b3f0-4f2c-82cf-fb2bbaab0984, kind: Service { id: 038e4c2b-2408-42f1-b025-c88f1167faaf }, name: Name("external-dns-038e4c2b-2408-42f1-b025-c88f1167faaf"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 226, 83])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: b98af94a-ceb1-41e9-82a5-9be8a66b4c85, kind: Service { id: eb4a98a4-3147-445f-9881-33f4dffeca66 }, name: Name("external-dns-eb4a98a4-3147-445f-9881-33f4dffeca66"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 148, 145])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 55f88ed5-95d7-43db-b06a-514b7adb4cdb, kind: Service { id: 7641195e-73b4-4972-ad51-4fcacb36a4be }, name: Name("nexus-7641195e-73b4-4972-ad51-4fcacb36a4be"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 205, 204])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 984220cf-9c9a-412c-a465-3579565ca87a, kind: Service { id: e181617d-520a-429d-85ef-385f6dde38ea }, name: Name("nexus-e181617d-520a-429d-85ef-385f6dde38ea"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 83])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9df31eec-63bd-49fd-99e9-b3a05e952553, kind: Service { id: 6c4893ca-1b18-4910-8b18-504e5fcb8dc5 }, name: Name("nexus-6c4893ca-1b18-4910-8b18-504e5fcb8dc5"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 162, 5])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
28952024-11-13T05:18:11.705ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
28962024-11-13T05:18:11.705ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte0" rules = []
28972024-11-13T05:18:11.705ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
28982024-11-13T05:18:11.705ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
28992024-11-13T05:18:11.705ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
29002024-11-13T05:18:11.705ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:750 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
29012024-11-13T05:18:11.705ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 323 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63415 req_id = 6658725f-ec6a-4a29-988c-d0b5e8071bc5 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
29022024-11-13T05:18:11.708ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29032024-11-13T05:18:11.708ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c9fdd7dd-67c5-4597-a857-03bc9769dc87
29042024-11-13T05:18:11.735ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29052024-11-13T05:18:11.735ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_27d5a04d-ccb1-4219-851b-abc94499936f
29062024-11-13T05:18:11.740ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
29072024-11-13T05:18:11.740ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 128247 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:49850 req_id = 3a0d2337-1bb5-4e09-a3be-b28753ea561e response_code = 204 uri = /switch-ports
29082024-11-13T05:18:11.746ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 639 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49850 req_id = 5630f27b-ad3e-4f62-b002-c072adbaaf4c response_code = 200 uri = /network-bootstore-config
29092024-11-13T05:18:11.750ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8e821c87-2d05-4e93-b160-49a645a177c5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29102024-11-13T05:18:11.750ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8e821c87-2d05-4e93-b160-49a645a177c5
29112024-11-13T05:18:11.773ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29122024-11-13T05:18:11.774ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c7851ba0-6692-4564-8a56-a39ae4767989
29132024-11-13T05:18:11.783ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
29142024-11-13T05:18:11.787ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: e770fce0-3dd5-4678-9037-68ea781dbb30 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29152024-11-13T05:18:11.787ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_e770fce0-3dd5-4678-9037-68ea781dbb30
29162024-11-13T05:18:11.805ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 313493 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37893 req_id = 5bf490cb-9f02-4c08-b6c0-65ff696fdd0d response_code = 200 uri = /inventory
29172024-11-13T05:18:11.812ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c2ce9d45-3d62-4234-a9db-c1d5c8884adc (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29182024-11-13T05:18:11.812ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c2ce9d45-3d62-4234-a9db-c1d5c8884adc
29192024-11-13T05:18:11.837ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c9fdd7dd-67c5-4597-a857-03bc9769dc87 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29202024-11-13T05:18:11.837ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c9fdd7dd-67c5-4597-a857-03bc9769dc87
29212024-11-13T05:18:11.846ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 192529 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:61667 req_id = b96e4c4e-45a3-49da-9553-106656ee0e15 response_code = 204 uri = /switch-ports
29222024-11-13T05:18:11.846ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 144 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61667 req_id = 4a5d826c-2e48-4b6a-a326-38580a70ea07 response_code = 200 uri = /network-bootstore-config
29232024-11-13T05:18:11.862ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 27d5a04d-ccb1-4219-851b-abc94499936f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29242024-11-13T05:18:11.862ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_27d5a04d-ccb1-4219-851b-abc94499936f
29252024-11-13T05:18:11.884ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c7851ba0-6692-4564-8a56-a39ae4767989 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
29262024-11-13T05:18:11.884ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_c7851ba0-6692-4564-8a56-a39ae4767989
29272024-11-13T05:18:11.897ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 300479 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44717 req_id = 67707e28-d27e-4f3c-aeec-af6c18e2b768 response_code = 200 uri = /inventory
29282024-11-13T05:18:19.306ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29292024-11-13T05:18:19.409ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29302024-11-13T05:18:19.410ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29312024-11-13T05:18:19.411ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29322024-11-13T05:18:19.411ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29332024-11-13T05:18:19.412ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29342024-11-13T05:18:19.413ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29352024-11-13T05:18:19.413ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29362024-11-13T05:18:19.415ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29372024-11-13T05:18:19.416ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29382024-11-13T05:18:19.416ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29392024-11-13T05:18:19.417ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29402024-11-13T05:18:19.417ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29412024-11-13T05:18:19.418ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29422024-11-13T05:18:19.418ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29432024-11-13T05:18:19.419ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29442024-11-13T05:18:19.419ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29452024-11-13T05:18:19.420ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29462024-11-13T05:18:24.803ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:45658
29472024-11-13T05:18:24.803ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 37 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:45658 req_id = cfc32896-5e08-43fd-b101-0eddc5e1b503 response_code = 200 uri = /baseboard
29482024-11-13T05:18:29.311ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29492024-11-13T05:18:29.412ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29502024-11-13T05:18:29.412ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29512024-11-13T05:18:29.413ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29522024-11-13T05:18:29.414ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29532024-11-13T05:18:29.414ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29542024-11-13T05:18:29.415ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29552024-11-13T05:18:29.416ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29562024-11-13T05:18:29.417ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29572024-11-13T05:18:29.418ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29582024-11-13T05:18:29.418ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29592024-11-13T05:18:29.419ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29602024-11-13T05:18:29.420ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29612024-11-13T05:18:29.420ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29622024-11-13T05:18:29.421ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29632024-11-13T05:18:29.421ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29642024-11-13T05:18:29.422ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29652024-11-13T05:18:29.422ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29662024-11-13T05:18:39.315ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29672024-11-13T05:18:39.414ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29682024-11-13T05:18:39.415ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29692024-11-13T05:18:39.415ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29702024-11-13T05:18:39.416ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29712024-11-13T05:18:39.417ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29722024-11-13T05:18:39.417ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29732024-11-13T05:18:39.418ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29742024-11-13T05:18:39.420ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29752024-11-13T05:18:39.420ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29762024-11-13T05:18:39.421ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29772024-11-13T05:18:39.421ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29782024-11-13T05:18:39.422ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29792024-11-13T05:18:39.422ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29802024-11-13T05:18:39.423ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29812024-11-13T05:18:39.423ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29822024-11-13T05:18:39.424ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29832024-11-13T05:18:39.424ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
29842024-11-13T05:18:40.027ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42393
29852024-11-13T05:18:40.027ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42393 req_id = 88a5ec32-2135-4b68-b78d-fa95a1365b67 response_code = 200 uri = /vpc-routes
29862024-11-13T05:18:40.084ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
29872024-11-13T05:18:40.084ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42393 req_id = 22fb230f-68e0-4f02-a4e4-6bf4bce5412c response_code = 204 uri = /eip-gateways
29882024-11-13T05:18:40.093ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 211 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42393 req_id = 32cf1e57-359d-4848-8884-6835cf971662 response_code = 204 uri = /vpc-routes
29892024-11-13T05:18:40.178ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47760
29902024-11-13T05:18:40.178ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 128 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47760 req_id = d3f84fe7-e844-4204-b9b0-3795b44b14e6 response_code = 200 uri = /v2p
29912024-11-13T05:18:40.213ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40968
29922024-11-13T05:18:40.213ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
29932024-11-13T05:18:40.255ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
29942024-11-13T05:18:40.316ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 102701 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:40968 req_id = a1a78bed-bae6-42b0-9e67-b2587a7125db response_code = 204 uri = /switch-ports
29952024-11-13T05:18:40.316ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40968 req_id = 084b2aa9-1697-485e-bc96-272c2edb1602 response_code = 200 uri = /network-bootstore-config
29962024-11-13T05:18:40.975ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55085
29972024-11-13T05:18:40.975ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55085 req_id = a18fc9de-02f0-4434-9021-ed5bb42de5c6 response_code = 200 uri = /v2p
29982024-11-13T05:18:41.033ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33500
29992024-11-13T05:18:41.033ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 71 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33500 req_id = cdb4799e-ab34-403a-9fb6-d97dd866a74e response_code = 200 uri = /vpc-routes
30002024-11-13T05:18:41.080ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
30012024-11-13T05:18:41.080ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:33500 req_id = 8bba89bf-6673-48d0-9a1e-db1548513ece response_code = 204 uri = /eip-gateways
30022024-11-13T05:18:41.090ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 195 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:33500 req_id = 9b3a1e60-4b2b-4b77-b909-688a960bf492 response_code = 204 uri = /vpc-routes
30032024-11-13T05:18:41.232ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65291
30042024-11-13T05:18:41.232ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65291 req_id = e828a055-3320-4623-ad25-aae6f8924efd response_code = 200 uri = /vpc-routes
30052024-11-13T05:18:41.280ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
30062024-11-13T05:18:41.280ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:65291 req_id = 3ecf1467-cc30-44cb-8cfa-814ae7b44da9 response_code = 204 uri = /eip-gateways
30072024-11-13T05:18:41.289ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 193 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:65291 req_id = 3fa7dca2-181e-4d6f-8550-d60e6cb37b9b response_code = 204 uri = /vpc-routes
30082024-11-13T05:18:41.290ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53835
30092024-11-13T05:18:41.290ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
30102024-11-13T05:18:41.331ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
30112024-11-13T05:18:41.391ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 101690 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:53835 req_id = 0814f6c7-1a33-41da-8189-32c2eda35299 response_code = 204 uri = /switch-ports
30122024-11-13T05:18:41.392ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51217
30132024-11-13T05:18:41.392ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59821
30142024-11-13T05:18:41.392ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
30152024-11-13T05:18:41.392ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 270 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53835 req_id = e178f3fc-b44e-4dbd-b329-0e1beee7031e response_code = 200 uri = /network-bootstore-config
30162024-11-13T05:18:41.396ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 4351 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51217 req_id = 6fb1fc1c-665a-444b-83a4-29d7c0c8f045 response_code = 200 uri = /v2p
30172024-11-13T05:18:41.429ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
30182024-11-13T05:18:41.488ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 96386 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59821 req_id = 29a8b45b-8fa5-4005-95db-9ea37f1182c4 response_code = 204 uri = /switch-ports
30192024-11-13T05:18:41.489ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59821 req_id = c4e5e9d8-17be-422c-a199-008a7a8ced2b response_code = 200 uri = /network-bootstore-config
30202024-11-13T05:18:49.318ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30212024-11-13T05:18:49.415ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30222024-11-13T05:18:49.416ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30232024-11-13T05:18:49.417ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30242024-11-13T05:18:49.418ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30252024-11-13T05:18:49.418ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30262024-11-13T05:18:49.419ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30272024-11-13T05:18:49.420ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30282024-11-13T05:18:49.421ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30292024-11-13T05:18:49.422ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30302024-11-13T05:18:49.422ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30312024-11-13T05:18:49.423ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30322024-11-13T05:18:49.424ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30332024-11-13T05:18:49.424ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30342024-11-13T05:18:49.425ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30352024-11-13T05:18:49.425ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30362024-11-13T05:18:49.425ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30372024-11-13T05:18:49.426ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30382024-11-13T05:18:54.837ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:40051
30392024-11-13T05:18:54.837ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 35 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:40051 req_id = 2617cdc5-76dd-4e69-aba0-59a7e6739906 response_code = 200 uri = /baseboard
30402024-11-13T05:18:59.322ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30412024-11-13T05:18:59.418ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30422024-11-13T05:18:59.419ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30432024-11-13T05:18:59.420ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30442024-11-13T05:18:59.421ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30452024-11-13T05:18:59.421ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30462024-11-13T05:18:59.422ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30472024-11-13T05:18:59.422ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30482024-11-13T05:18:59.423ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30492024-11-13T05:18:59.424ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30502024-11-13T05:18:59.424ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30512024-11-13T05:18:59.425ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30522024-11-13T05:18:59.425ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30532024-11-13T05:18:59.426ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30542024-11-13T05:18:59.426ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30552024-11-13T05:18:59.427ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30562024-11-13T05:18:59.427ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30572024-11-13T05:18:59.428ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30582024-11-13T05:19:09.325ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30592024-11-13T05:19:09.421ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30602024-11-13T05:19:09.422ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30612024-11-13T05:19:09.423ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30622024-11-13T05:19:09.424ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30632024-11-13T05:19:09.424ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30642024-11-13T05:19:09.425ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30652024-11-13T05:19:09.426ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30662024-11-13T05:19:09.426ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30672024-11-13T05:19:09.427ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30682024-11-13T05:19:09.427ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30692024-11-13T05:19:09.428ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30702024-11-13T05:19:09.429ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30712024-11-13T05:19:09.429ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30722024-11-13T05:19:09.430ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30732024-11-13T05:19:09.430ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30742024-11-13T05:19:09.431ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30752024-11-13T05:19:09.431ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
30762024-11-13T05:19:10.187ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61785
30772024-11-13T05:19:10.188ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61785 req_id = 672fb1ff-7da8-42ca-ab45-c80f5d5e1b94 response_code = 200 uri = /vpc-routes
30782024-11-13T05:19:10.202ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
30792024-11-13T05:19:10.202ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61785 req_id = dae8973a-eaaa-4157-8186-e5545ea97368 response_code = 204 uri = /eip-gateways
30802024-11-13T05:19:10.211ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 174 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61785 req_id = 7c49483d-842c-4bbf-b28b-bcb99cc745ee response_code = 204 uri = /vpc-routes
30812024-11-13T05:19:10.365ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43804
30822024-11-13T05:19:10.365ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43804 req_id = 306b84b4-f73f-4f9e-bd88-aed8fd7dc65c response_code = 200 uri = /v2p
30832024-11-13T05:19:10.445ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55250
30842024-11-13T05:19:10.445ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
30852024-11-13T05:19:10.483ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
30862024-11-13T05:19:10.539ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 94038 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:55250 req_id = 30380248-9697-4a58-bf68-519243dcd0dc response_code = 204 uri = /switch-ports
30872024-11-13T05:19:10.539ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55250 req_id = 597313b6-1f98-4fac-8a8b-dd2a8f4c3aa2 response_code = 200 uri = /network-bootstore-config
30882024-11-13T05:19:11.138ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43787
30892024-11-13T05:19:11.138ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43787 req_id = 53f10112-2ce0-4630-a72f-04b285bc601d response_code = 200 uri = /vpc-routes
30902024-11-13T05:19:11.184ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
30912024-11-13T05:19:11.184ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43787 req_id = 255fe227-430c-4afe-92e0-1c93b3d9fe9b response_code = 204 uri = /eip-gateways
30922024-11-13T05:19:11.193ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 207 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43787 req_id = 21a2e7ee-9cb6-4748-a327-5c5a7811cfab response_code = 204 uri = /vpc-routes
30932024-11-13T05:19:11.251ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45981
30942024-11-13T05:19:11.251ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 34 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45981 req_id = e4f1bcf8-d3a7-4a09-aae7-088e48bbadd3 response_code = 200 uri = /vpc-routes
30952024-11-13T05:19:11.267ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
30962024-11-13T05:19:11.267ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 43 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45981 req_id = b3f8500d-dea3-4dad-8bc3-608778b63862 response_code = 204 uri = /eip-gateways
30972024-11-13T05:19:11.328ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34167
30982024-11-13T05:19:11.329ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34167 req_id = ea2c9eb0-1725-4caa-adb3-36897498a1ab response_code = 200 uri = /v2p
30992024-11-13T05:19:11.344ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64223
31002024-11-13T05:19:11.344ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64223 req_id = 54b4139a-2751-41ca-9291-6538567c74f7 response_code = 200 uri = /v2p
31012024-11-13T05:19:11.353ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 172 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45981 req_id = 88a1fe8a-b101-4e96-b7f0-b817e946dbf0 response_code = 204 uri = /vpc-routes
31022024-11-13T05:19:11.378ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42492
31032024-11-13T05:19:11.378ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
31042024-11-13T05:19:11.419ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
31052024-11-13T05:19:11.479ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 101074 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:42492 req_id = 186cb5d3-6ec7-48e5-a0c5-34971d9426fe response_code = 204 uri = /switch-ports
31062024-11-13T05:19:11.480ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58080
31072024-11-13T05:19:11.480ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 147 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42492 req_id = 2108d067-18e2-4505-8c63-3211f131e58a response_code = 200 uri = /network-bootstore-config
31082024-11-13T05:19:11.480ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
31092024-11-13T05:19:11.522ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
31102024-11-13T05:19:11.582ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 102236 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:58080 req_id = d63ba72f-c54f-4c78-a709-5369d1548d2c response_code = 204 uri = /switch-ports
31112024-11-13T05:19:11.605ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58080 req_id = c7ce100d-a2a9-4f9b-85c9-c09e80feae7c response_code = 200 uri = /network-bootstore-config
31122024-11-13T05:19:19.329ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31132024-11-13T05:19:19.425ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31142024-11-13T05:19:19.426ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31152024-11-13T05:19:19.426ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31162024-11-13T05:19:19.427ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31172024-11-13T05:19:19.428ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31182024-11-13T05:19:19.428ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31192024-11-13T05:19:19.429ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31202024-11-13T05:19:19.430ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31212024-11-13T05:19:19.430ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31222024-11-13T05:19:19.431ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31232024-11-13T05:19:19.432ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31242024-11-13T05:19:19.432ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31252024-11-13T05:19:19.433ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31262024-11-13T05:19:19.433ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31272024-11-13T05:19:19.434ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31282024-11-13T05:19:19.434ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31292024-11-13T05:19:19.435ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31302024-11-13T05:19:24.871ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:44549
31312024-11-13T05:19:24.871ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 35 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:44549 req_id = 8ca3fad6-b2a5-40b3-b45d-17198f38e039 response_code = 200 uri = /baseboard
31322024-11-13T05:19:29.334ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31332024-11-13T05:19:29.427ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31342024-11-13T05:19:29.428ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31352024-11-13T05:19:29.429ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31362024-11-13T05:19:29.429ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31372024-11-13T05:19:29.430ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31382024-11-13T05:19:29.430ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31392024-11-13T05:19:29.431ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31402024-11-13T05:19:29.432ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31412024-11-13T05:19:29.432ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31422024-11-13T05:19:29.433ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31432024-11-13T05:19:29.433ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31442024-11-13T05:19:29.434ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31452024-11-13T05:19:29.434ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31462024-11-13T05:19:29.435ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31472024-11-13T05:19:29.435ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31482024-11-13T05:19:29.436ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31492024-11-13T05:19:29.436ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31502024-11-13T05:19:39.337ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31512024-11-13T05:19:39.429ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31522024-11-13T05:19:39.430ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31532024-11-13T05:19:39.431ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31542024-11-13T05:19:39.431ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31552024-11-13T05:19:39.432ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31562024-11-13T05:19:39.433ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31572024-11-13T05:19:39.433ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31582024-11-13T05:19:39.434ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31592024-11-13T05:19:39.435ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31602024-11-13T05:19:39.435ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31612024-11-13T05:19:39.436ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31622024-11-13T05:19:39.436ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31632024-11-13T05:19:39.437ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31642024-11-13T05:19:39.437ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31652024-11-13T05:19:39.438ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31662024-11-13T05:19:39.438ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31672024-11-13T05:19:39.439ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
31682024-11-13T05:19:40.111ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42171
31692024-11-13T05:19:40.111ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42171 req_id = a4cb2cb4-ff57-4716-925e-39f2287ea96d response_code = 200 uri = /vpc-routes
31702024-11-13T05:19:40.159ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
31712024-11-13T05:19:40.159ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42171 req_id = 79b730b0-9259-4a01-afb8-40f7010e21d0 response_code = 204 uri = /eip-gateways
31722024-11-13T05:19:40.199ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48071
31732024-11-13T05:19:40.199ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48071 req_id = f7184bea-4bfc-4ef0-ad74-987598bb08af response_code = 200 uri = /v2p
31742024-11-13T05:19:40.206ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42171 req_id = 0c293196-3256-4b9d-87b5-6518c3ef7030 response_code = 204 uri = /vpc-routes
31752024-11-13T05:19:40.272ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35165
31762024-11-13T05:19:40.272ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
31772024-11-13T05:19:40.317ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
31782024-11-13T05:19:40.382ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 110465 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:35165 req_id = c64b1082-077b-474c-b09e-bc3940d8aa95 response_code = 204 uri = /switch-ports
31792024-11-13T05:19:40.382ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35165 req_id = 8964454c-576b-48ea-ba63-c14aac41b8d6 response_code = 200 uri = /network-bootstore-config
31802024-11-13T05:19:41.034ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42291
31812024-11-13T05:19:41.034ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42291 req_id = 83d1e6ff-e99c-4d9e-a730-fa85b3e90ea1 response_code = 200 uri = /vpc-routes
31822024-11-13T05:19:41.048ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
31832024-11-13T05:19:41.048ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42291 req_id = 4f50adaf-ddee-465e-ab49-d2241290b83c response_code = 204 uri = /eip-gateways
31842024-11-13T05:19:41.063ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 247 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42291 req_id = 4ffaf668-40d0-4701-ace1-aa8d4377cd44 response_code = 204 uri = /vpc-routes
31852024-11-13T05:19:41.104ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47049
31862024-11-13T05:19:41.104ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47049 req_id = 76c5c563-23f6-441f-863e-a083c6a7b0d3 response_code = 200 uri = /v2p
31872024-11-13T05:19:41.173ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60581
31882024-11-13T05:19:41.173ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
31892024-11-13T05:19:41.217ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
31902024-11-13T05:19:41.287ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 113796 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:60581 req_id = bc401df8-7189-43d4-b5f2-27ced9be1c55 response_code = 204 uri = /switch-ports
31912024-11-13T05:19:41.287ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 160 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60581 req_id = 0961fc55-3736-42a9-8d0a-c2e626c89ada response_code = 200 uri = /network-bootstore-config
31922024-11-13T05:19:41.580ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40574
31932024-11-13T05:19:41.581ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40574 req_id = 4bf78daa-e9d5-48e4-9bb7-888a12114e32 response_code = 200 uri = /vpc-routes
31942024-11-13T05:19:41.627ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
31952024-11-13T05:19:41.627ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40574 req_id = 2cf265aa-5f7a-49f0-9fd8-56fec902b635 response_code = 204 uri = /eip-gateways
31962024-11-13T05:19:41.643ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60720
31972024-11-13T05:19:41.643ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60720 req_id = 4b4a3afe-ed30-4b81-918e-6d5bc34a8e36 response_code = 200 uri = /v2p
31982024-11-13T05:19:41.651ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40574 req_id = 22696984-2719-4c86-9258-f82094776830 response_code = 204 uri = /vpc-routes
31992024-11-13T05:19:41.657ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46043
32002024-11-13T05:19:41.657ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
32012024-11-13T05:19:41.704ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
32022024-11-13T05:19:41.772ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 114309 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:46043 req_id = ab1145b8-d43f-499b-8376-368ee2b8bbd4 response_code = 204 uri = /switch-ports
32032024-11-13T05:19:41.772ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46043 req_id = e1011cfd-ccd5-4735-81f0-fb0842dc1452 response_code = 200 uri = /network-bootstore-config
32042024-11-13T05:19:46.030ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44359
32052024-11-13T05:19:46.030ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44359 req_id = 291e5255-9f55-4fbe-9575-34456165ae18 response_code = 200 uri = /v2p
32062024-11-13T05:19:46.030ZINFOSledAgent (PortManager): Mapping virtual NIC to physical host file = illumos-utils/src/opte/port_manager.rs:838 mapping = VirtualNetworkInterfaceHost { virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), physical_host_ip: fd00:1122:3344:101::1, vni: Vni(3170179) }
32072024-11-13T05:19:46.030ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44359 req_id = b98b6781-cfdb-4869-a0c2-e52e63ed3838 response_code = 204 uri = /v2p
32082024-11-13T05:19:46.153ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62930
32092024-11-13T05:19:46.158ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:555 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 0e128a6f-fc83-417f-ab6b-0a6bb68d82c1, kind: Instance { id: 3bddec9b-236c-407a-ae26-63052a5ace8c }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(3170179), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0e128a6f-fc83-417f-ab6b-0a6bb68d82c1, kind: Instance { id: 3bddec9b-236c-407a-ae26-63052a5ace8c }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(3170179), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0e128a6f-fc83-417f-ab6b-0a6bb68d82c1, kind: Instance { id: 3bddec9b-236c-407a-ae26-63052a5ace8c }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(3170179), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(3170179))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0e128a6f-fc83-417f-ab6b-0a6bb68d82c1, kind: Instance { id: 3bddec9b-236c-407a-ae26-63052a5ace8c }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(3170179), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-26bb32eb570", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 8c8c3371-8eb1-4dd5-9476-2ad2952772e5, read_only_parent: Some(Volume { block_size: 512, id: d18fcaad-23cb-4652-9e34-bfbdae05c312, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 216e5d18-430d-4231-b12d-d985e3529450, key: Some("PXTsWgI+zDZFioMRVo6NWZ3Ckvm9x2e7DiLr7Obgf4o="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::19]:19002", "[fd00:1122:3344:101::13]:19001", "[fd00:1122:3344:101::14]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 8c8c3371-8eb1-4dd5-9476-2ad2952772e5, key: Some("9UEiJFsQu+9Vp/utH9pmxvHTpjHfEEcqCokdbKEcibY="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::17]:19001", "[fd00:1122:3344:101::19]:19003", "[fd00:1122:3344:101::16]:19000"] } }] } }], boot_settings: Some(BootSettings { order: [BootOrderEntry { name: "disk-26bb32eb570" }] }), cloud_init_bytes: Some(..) } instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c metadata = InstanceMetadata { silo_id: 26cfece8-d05c-41eb-b6a0-db2c20fff7a5, project_id: e384052f-88fa-4666-ac34-dbd6b201f32f } migration_id = None propolis_addr = [fd00:1122:3344:101::1:0]:12400 propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10 vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-11-13T05:19:46.153682196Z }
32102024-11-13T05:19:46.158ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:588 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c migration_id = None propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32112024-11-13T05:19:46.158ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:1281 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c migration_id = None propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10 state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 0e128a6f-fc83-417f-ab6b-0a6bb68d82c1, kind: Instance { id: 3bddec9b-236c-407a-ae26-63052a5ace8c }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(3170179), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0e128a6f-fc83-417f-ab6b-0a6bb68d82c1, kind: Instance { id: 3bddec9b-236c-407a-ae26-63052a5ace8c }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(3170179), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0e128a6f-fc83-417f-ab6b-0a6bb68d82c1, kind: Instance { id: 3bddec9b-236c-407a-ae26-63052a5ace8c }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(3170179), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(3170179))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0e128a6f-fc83-417f-ab6b-0a6bb68d82c1, kind: Instance { id: 3bddec9b-236c-407a-ae26-63052a5ace8c }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(3170179), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-26bb32eb570", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 8c8c3371-8eb1-4dd5-9476-2ad2952772e5, read_only_parent: Some(Volume { block_size: 512, id: d18fcaad-23cb-4652-9e34-bfbdae05c312, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 216e5d18-430d-4231-b12d-d985e3529450, key: Some("PXTsWgI+zDZFioMRVo6NWZ3Ckvm9x2e7DiLr7Obgf4o="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::19]:19002", "[fd00:1122:3344:101::13]:19001", "[fd00:1122:3344:101::14]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 8c8c3371-8eb1-4dd5-9476-2ad2952772e5, key: Some("9UEiJFsQu+9Vp/utH9pmxvHTpjHfEEcqCokdbKEcibY="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::17]:19001", "[fd00:1122:3344:101::19]:19003", "[fd00:1122:3344:101::16]:19000"] } }] } }], boot_settings: Some(BootSettings { order: [BootOrderEntry { name: "disk-26bb32eb570" }] }), cloud_init_bytes: Some(..) }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-11-13T05:19:46.153682196Z }, propolis_addr: [fd00:1122:3344:101::1:0]:12400, migration_id: None }
32122024-11-13T05:19:46.159ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 5112 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:62930 req_id = b8f83730-623b-462c-ab2a-cbef9d622a61 response_code = 200 uri = /vmms/0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32132024-11-13T05:19:46.234ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48087
32142024-11-13T05:19:46.235ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 855 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48087 req_id = 32ca03f6-c8e2-4d0d-a5be-2e3958296077 response_code = 200 uri = /vmms/0c90c4af-fcbc-45e0-82c8-050aac5b3b10/state
32152024-11-13T05:19:46.236ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:430 port = Port { inner: PortInner(PortData { name: "opte6", ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 240, 0, 0]), slot: 0, vni: Vni { inner: 3170179 }, subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 } }) }
32162024-11-13T05:19:46.237ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
32172024-11-13T05:19:46.279ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10 file = illumos-utils/src/zone.rs:325 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32182024-11-13T05:19:46.301ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10 file = illumos-utils/src/zone.rs:363 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32192024-11-13T05:19:46.308ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61538
32202024-11-13T05:19:46.308ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 68 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61538 req_id = 09990140-bd01-4f1d-9021-6575a656903c response_code = 200 uri = /vpc-routes
32212024-11-13T05:19:46.323ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {0e128a6f-fc83-417f-ab6b-0a6bb68d82c1: {10.151.2.175: {674688f5-6813-4eba-92f7-85ba7020e4ea}, 10.151.2.176: {674688f5-6813-4eba-92f7-85ba7020e4ea}}} }
32222024-11-13T05:19:47.812ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlInstance0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1:0'/> <propval type="astring" name="listen_port" value='12400'/> <propval type="astring" name="metric_addr" value='dns'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32232024-11-13T05:19:47.813ZINFOSledAgent (InstanceManager): Booting oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10 zone file = illumos-utils/src/running_zone.rs:488 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10 zone = oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32242024-11-13T05:19:48.078ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10") failed: Property not found. retry in 67.298542ms file = illumos-utils/src/svc.rs:36 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10 zone = oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32252024-11-13T05:19:48.170ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10") failed: Property not found. retry in 79.669962ms file = illumos-utils/src/svc.rs:36 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10 zone = oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32262024-11-13T05:19:48.279ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10") failed: Property not found. retry in 202.220328ms file = illumos-utils/src/svc.rs:36 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10 zone = oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32272024-11-13T05:19:48.494ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10") failed: Property not found. retry in 385.707668ms file = illumos-utils/src/svc.rs:36 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10 zone = oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32282024-11-13T05:19:48.900ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10") failed: Property not found. retry in 512.765344ms file = illumos-utils/src/svc.rs:36 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10 zone = oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32292024-11-13T05:19:49.342ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32302024-11-13T05:19:49.428ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10") failed: Property not found. retry in 651.372702ms file = illumos-utils/src/svc.rs:36 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10 zone = oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32312024-11-13T05:19:49.433ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32322024-11-13T05:19:49.434ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32332024-11-13T05:19:49.435ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32342024-11-13T05:19:49.435ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32352024-11-13T05:19:49.436ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32362024-11-13T05:19:49.437ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32372024-11-13T05:19:49.437ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32382024-11-13T05:19:49.439ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32392024-11-13T05:19:49.439ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32402024-11-13T05:19:49.440ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32412024-11-13T05:19:49.440ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32422024-11-13T05:19:49.441ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32432024-11-13T05:19:49.441ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32442024-11-13T05:19:49.443ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32452024-11-13T05:19:49.443ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32462024-11-13T05:19:49.444ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32472024-11-13T05:19:49.444ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32482024-11-13T05:19:50.093ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10") failed: Property not found. retry in 749.620238ms file = illumos-utils/src/svc.rs:36 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10 zone = oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32492024-11-13T05:19:50.856ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10") failed: Property not found. retry in 1.37709087s file = illumos-utils/src/svc.rs:36 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10 zone = oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32502024-11-13T05:19:52.247ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10") failed: Property not found. retry in 1.000920609s file = illumos-utils/src/svc.rs:36 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10 zone = oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32512024-11-13T05:19:53.284ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10 file = sled-agent/src/instance.rs:1787 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32522024-11-13T05:19:53.300ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10") failed: Property not found. retry in 65.589311ms file = illumos-utils/src/svc.rs:36 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32532024-11-13T05:19:53.384ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10") failed: Property not found. retry in 111.691381ms file = illumos-utils/src/svc.rs:36 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32542024-11-13T05:19:53.511ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:1796 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32552024-11-13T05:19:53.547ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:1820 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32562024-11-13T05:19:53.694ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:932 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32572024-11-13T05:19:53.697ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1231 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32582024-11-13T05:19:53.697ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Creating), migration_in: None, migration_out: None, time: 2024-11-13T05:19:53.697967719Z } file = sled-agent/src/instance.rs:780 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32592024-11-13T05:19:53.698ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-11-13T05:19:53.697967719Z } propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32602024-11-13T05:19:53.698ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10 state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-11-13T05:19:53.697967719Z }, migration_in: None, migration_out: None }
32612024-11-13T05:19:53.698ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 7374527 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:61538 req_id = cfdeb1a5-5fc5-4702-af52-d0c8d85afc9c response_code = 204 uri = /eip-gateways
32622024-11-13T05:19:53.736ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_in: None, migration_out: None, time: 2024-11-13T05:19:53.736108433Z } file = sled-agent/src/instance.rs:780 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32632024-11-13T05:19:53.736ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-11-13T05:19:53.736108433Z } propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32642024-11-13T05:19:53.736ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-11-13T05:19:53.736108433Z }, migration_in: None, migration_out: None }
32652024-11-13T05:19:53.786ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 416 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:61538 req_id = 5f2d4616-b61f-4a79-8317-91070315c0fc response_code = 204 uri = /vpc-routes
32662024-11-13T05:19:54.904ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:65187
32672024-11-13T05:19:54.904ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 65 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:65187 req_id = 9eb5970b-f417-4040-ae31-22e0d52c242b response_code = 200 uri = /baseboard
32682024-11-13T05:19:59.346ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32692024-11-13T05:19:59.435ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32702024-11-13T05:19:59.436ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32712024-11-13T05:19:59.437ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32722024-11-13T05:19:59.438ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32732024-11-13T05:19:59.438ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32742024-11-13T05:19:59.439ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32752024-11-13T05:19:59.439ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32762024-11-13T05:19:59.440ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32772024-11-13T05:19:59.441ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32782024-11-13T05:19:59.442ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32792024-11-13T05:19:59.443ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32802024-11-13T05:19:59.444ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32812024-11-13T05:19:59.444ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32822024-11-13T05:19:59.445ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32832024-11-13T05:19:59.445ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32842024-11-13T05:19:59.446ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32852024-11-13T05:19:59.446ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32862024-11-13T05:20:03.761ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44446
32872024-11-13T05:20:03.762ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 591 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44446 req_id = 17609c7e-a891-45f1-a4dd-a42c3ea85790 response_code = 200 uri = /vmms/0c90c4af-fcbc-45e0-82c8-050aac5b3b10/state
32882024-11-13T05:20:03.762ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_in: None, migration_out: None, time: 2024-11-13T05:20:03.762612484Z } file = sled-agent/src/instance.rs:780 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32892024-11-13T05:20:03.762ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-11-13T05:20:03.762612484Z } propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32902024-11-13T05:20:03.762ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10 state = SledVmmState { vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-11-13T05:20:03.762612484Z }, migration_in: None, migration_out: None }
32912024-11-13T05:20:03.786ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_in: None, migration_out: None, time: 2024-11-13T05:20:03.786296481Z } file = sled-agent/src/instance.rs:780 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32922024-11-13T05:20:03.786ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-11-13T05:20:03.786296481Z } propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32932024-11-13T05:20:03.786ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:816 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32942024-11-13T05:20:03.786ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
32952024-11-13T05:20:03.786ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:602 id = 15309069475637975175 n_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32962024-11-13T05:20:03.786ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:602 id = 8437640861448840183 n_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
32972024-11-13T05:20:03.786ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
32982024-11-13T05:20:03.786ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/147a026f-d4f7-4267-9015-fa2814ab1bfc/debug/bundle/zone", "/pool/int/3c73febf-14eb-49a3-9af1-023b1a81de43/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/8511034a-a894-457f-9cbc-98e6461f9f5a/crypt/debug", "/pool/ext/c867bbf7-921c-4db1-887a-4002bcc6b446/crypt/debug", "/pool/ext/204f40c4-4e09-4726-a368-a3da2ac1b831/crypt/debug", "/pool/ext/8e821c87-2d05-4e93-b160-49a645a177c5/crypt/debug", "/pool/ext/e770fce0-3dd5-4678-9037-68ea781dbb30/crypt/debug", "/pool/ext/c2ce9d45-3d62-4234-a9db-c1d5c8884adc/crypt/debug", "/pool/ext/c9fdd7dd-67c5-4597-a857-03bc9769dc87/crypt/debug", "/pool/ext/27d5a04d-ccb1-4219-851b-abc94499936f/crypt/debug", "/pool/ext/c7851ba0-6692-4564-8a56-a39ae4767989/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:339 zone_name = oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10
32992024-11-13T05:20:04.443ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1142 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10", bundle_id: d2aa5748-4be0-4732-9ee3-33a4fa22b6ee }, time_created: 2024-11-13T05:20:03.787496251Z, version: 0, cause: TerminatedInstance }
33002024-11-13T05:20:04.443ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_0c90c4af-fcbc-45e0-82c8-050aac5b3b10 file = sled-agent/src/instance.rs:1032 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
33012024-11-13T05:20:06.203ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:279 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
33022024-11-13T05:20:06.220ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:964 id = RouterId { vni: Vni(3170179), kind: System }
33032024-11-13T05:20:06.220ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:964 id = RouterId { vni: Vni(3170179), kind: Custom(V4(Ipv4Net { addr: 172.30.0.0, width: 22 })) }
33042024-11-13T05:20:06.220ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-13T05:20:06.220429317Z }, migration_in: None, migration_out: None }
33052024-11-13T05:20:06.274ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-11-13T05:20:06.220429317Z }, migration_in: None, migration_out: None }
33062024-11-13T05:20:06.277ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:969 instance_id = 3bddec9b-236c-407a-ae26-63052a5ace8c propolis_id = 0c90c4af-fcbc-45e0-82c8-050aac5b3b10
33072024-11-13T05:20:06.805ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57018
33082024-11-13T05:20:06.806ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57018 req_id = d8c968ce-96c7-447c-b629-5e0bd64c956c response_code = 200 uri = /vpc-routes
33092024-11-13T05:20:06.854ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
33102024-11-13T05:20:06.854ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1231 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33112024-11-13T05:20:06.855ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57018 req_id = e3847a84-9688-493a-96a2-094e54e7461f response_code = 204 uri = /eip-gateways
33122024-11-13T05:20:06.864ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 181 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57018 req_id = d0c8389c-8158-48cc-b28b-4ec156c560f4 response_code = 204 uri = /vpc-routes
33132024-11-13T05:20:06.936ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61536
33142024-11-13T05:20:06.936ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61536 req_id = c2f978ce-49f2-4301-9e13-46d163fe062c response_code = 200 uri = /v2p
33152024-11-13T05:20:08.033ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53971
33162024-11-13T05:20:08.033ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53971 req_id = 28cbb646-386b-45d6-b7aa-a59516b91e02 response_code = 200 uri = /v2p
33172024-11-13T05:20:08.144ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56739
33182024-11-13T05:20:08.144ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56739 req_id = 2d406dbc-0200-4581-8b9d-1f2973b4ad43 response_code = 200 uri = /vpc-routes
33192024-11-13T05:20:08.157ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
33202024-11-13T05:20:08.157ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56739 req_id = b198b6d5-6d8e-420d-9639-9d1725d2b3b7 response_code = 204 uri = /eip-gateways
33212024-11-13T05:20:08.165ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56739 req_id = fc651afe-7868-4514-b710-fc91c0ac2295 response_code = 204 uri = /vpc-routes
33222024-11-13T05:20:08.763ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57609
33232024-11-13T05:20:08.764ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57609 req_id = 291c63f2-aaae-48ef-8101-800129d16a76 response_code = 200 uri = /vpc-routes
33242024-11-13T05:20:08.776ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
33252024-11-13T05:20:08.776ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 52 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57609 req_id = c2b49500-d315-4504-b45b-24100b7594dc response_code = 204 uri = /eip-gateways
33262024-11-13T05:20:08.784ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 172 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57609 req_id = 3644fb08-4bd1-45e9-9202-17ce1a2f92fd response_code = 204 uri = /vpc-routes
33272024-11-13T05:20:09.350ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33282024-11-13T05:20:09.437ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33292024-11-13T05:20:09.438ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33302024-11-13T05:20:09.438ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33312024-11-13T05:20:09.439ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33322024-11-13T05:20:09.440ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33332024-11-13T05:20:09.441ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33342024-11-13T05:20:09.441ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33352024-11-13T05:20:09.442ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33362024-11-13T05:20:09.443ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33372024-11-13T05:20:09.444ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33382024-11-13T05:20:09.445ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33392024-11-13T05:20:09.446ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33402024-11-13T05:20:09.446ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33412024-11-13T05:20:09.447ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33422024-11-13T05:20:09.447ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33432024-11-13T05:20:09.448ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33442024-11-13T05:20:09.448ZWARNSledAgent (kstat-sampler-worker): sample queue is too full, dropping oldest samples file = oximeter/instruments/src/kstat/sampler.rs:688 n_current_samples = 500 n_new_samples = 6 n_overflow_samples = 6 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1
33452024-11-13T05:20:10.241ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60255
33462024-11-13T05:20:10.241ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60255 req_id = b7cba2be-968d-4ba3-8386-76a6687b80c7 response_code = 200 uri = /vpc-routes
33472024-11-13T05:20:10.257ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
33482024-11-13T05:20:10.257ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 68 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:60255 req_id = 5a4c1151-c185-4f20-8c69-089e8725d217 response_code = 204 uri = /eip-gateways
33492024-11-13T05:20:10.266ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 195 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:60255 req_id = 53331f03-0b24-4d94-9222-e44b7ef8156d response_code = 204 uri = /vpc-routes
33502024-11-13T05:20:10.323ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48775
33512024-11-13T05:20:10.323ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48775 req_id = 5968466a-b838-4842-8db4-678e0266d531 response_code = 200 uri = /v2p
33522024-11-13T05:20:10.439ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56701
33532024-11-13T05:20:10.440ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
33542024-11-13T05:20:10.482ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
33552024-11-13T05:20:10.541ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 101321 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:56701 req_id = 3926d55c-03fa-42ca-b664-1e055edac940 response_code = 204 uri = /switch-ports
33562024-11-13T05:20:10.541ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56701 req_id = e1cf3e3e-4867-4adf-afd0-a983da008767 response_code = 200 uri = /network-bootstore-config
33572024-11-13T05:20:11.112ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37115
33582024-11-13T05:20:11.112ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 72 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37115 req_id = adad64f9-c814-4265-846f-abe475ed2ece response_code = 200 uri = /vpc-routes
33592024-11-13T05:20:11.154ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45131
33602024-11-13T05:20:11.154ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45131 req_id = 71bba7e8-fe4b-4225-8aaf-e4a08f0d0542 response_code = 200 uri = /vpc-routes
33612024-11-13T05:20:11.173ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
33622024-11-13T05:20:11.173ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37115 req_id = 32f0c172-2f37-4144-a6dc-e96452bff852 response_code = 204 uri = /eip-gateways
33632024-11-13T05:20:11.184ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 194 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37115 req_id = bf9abb24-49e1-47b1-9833-9941e01ad806 response_code = 204 uri = /vpc-routes
33642024-11-13T05:20:11.237ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1205 sled_id = 4d997277-f640-48d7-af56-6df261d5afe1 values = ExternalIpGatewayMap { mappings: {} }
33652024-11-13T05:20:11.237ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45131 req_id = 6c3520dd-88a9-4d66-b8a7-b934d4554947 response_code = 204 uri = /eip-gateways
33662024-11-13T05:20:11.246ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 228 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45131 req_id = cc930ca1-318d-4391-8396-1846d135d72f response_code = 204 uri = /vpc-routes
33672024-11-13T05:20:11.265ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54814
33682024-11-13T05:20:11.266ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 163 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54814 req_id = a0f9d374-861a-4878-9167-5b0bf72c6dfd response_code = 200 uri = /v2p
33692024-11-13T05:20:11.322ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61378
33702024-11-13T05:20:11.322ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
33712024-11-13T05:20:11.368ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
33722024-11-13T05:20:11.429ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 107170 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:61378 req_id = 9bbdea82-b3d3-4003-ba0c-22c311c95ff5 response_code = 204 uri = /switch-ports
33732024-11-13T05:20:11.429ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46039
33742024-11-13T05:20:11.429ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46039 req_id = 9f3e0ae1-7fed-43c0-b100-bcfdee97e125 response_code = 200 uri = /v2p
33752024-11-13T05:20:11.430ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61378 req_id = 4aa45fe0-d7d6-4baf-a27b-deec85b7b30f response_code = 200 uri = /network-bootstore-config
33762024-11-13T05:20:11.536ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:1105 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54507
33772024-11-13T05:20:11.537ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
33782024-11-13T05:20:11.579ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
33792024-11-13T05:20:11.638ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 101034 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:54507 req_id = 3115e969-a61b-4a9e-b3a3-469be0ed63ef response_code = 204 uri = /switch-ports
33802024-11-13T05:20:11.638ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.12.0/src/server.rs:950 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54507 req_id = 483283b0-9109-46f9-8087-3b3e17f26e70 response_code = 200 uri = /network-bootstore-config