Max level shown:
1 [ Jan 8 18:19:54 Enabled. ]
2 [ Jan 8 18:19:54 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Jan 8 18:19:54 Method "start" exited with status 0. ]
4 [ Jan 8 18:19:54 Rereading configuration. ]
5 [ Jan 8 18:19:54 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72025-01-08T18:19:54.185ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:113
82025-01-08T18:19:54.185ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:255
92025-01-08T18:19:54.238ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:262
102025-01-08T18:19:54.238ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
112025-01-08T18:19:54.354ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
122025-01-08T18:19:54.363ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
132025-01-08T18:19:54.371ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:272 path = /var/run/oxide/
142025-01-08T18:19:54.974ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fdb0:18c0:4d0d:9fb2::, width: 64 }
152025-01-08T18:19:55.114ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:126
162025-01-08T18:19:55.114ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:138
172025-01-08T18:19:55.115ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:151
182025-01-08T18:19:55.115ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:173 nongimlet_observed_disks = [] sled_mode = Scrimlet { asic: SoftNpuZone }
192025-01-08T18:19:55.115ZINFOSledAgent (DumpSetup-worker): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:520
202025-01-08T18:19:55.115ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:717
212025-01-08T18:19:55.118ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "buskin", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:624
222025-01-08T18:19:55.118ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:766
232025-01-08T18:19:55.118ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:187
242025-01-08T18:19:55.118ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:825
252025-01-08T18:19:55.118ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_0.vdev
262025-01-08T18:19:55.118ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = None
272025-01-08T18:19:55.118ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
282025-01-08T18:19:55.118ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
292025-01-08T18:19:55.118ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } path = /scratch/m2_0.vdev
302025-01-08T18:19:55.129ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_0.vdev file = sled-hardware/src/disk.rs:367
312025-01-08T18:19:55.129ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = e4d4e46e-f335-4351-a367-c84fc95a9cd2 (zpool)
322025-01-08T18:19:55.226ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: e4d4e46e-f335-4351-a367-c84fc95a9cd2 (zpool), kind: Internal }
332025-01-08T18:19:55.588ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:607
342025-01-08T18:19:55.588ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_1.vdev
352025-01-08T18:19:55.588ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
362025-01-08T18:19:55.588ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
372025-01-08T18:19:55.588ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
382025-01-08T18:19:55.588ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
392025-01-08T18:19:55.588ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } path = /scratch/m2_1.vdev
402025-01-08T18:19:55.597ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_1.vdev file = sled-hardware/src/disk.rs:367
412025-01-08T18:19:55.597ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = 74c5b5be-8c07-4870-ad96-3bd00809d0ec (zpool)
422025-01-08T18:19:55.686ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 74c5b5be-8c07-4870-ad96-3bd00809d0ec (zpool), kind: Internal }
432025-01-08T18:19:56.048ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_0.vdev
442025-01-08T18:19:56.048ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: e4d4e46e-f335-4351-a367-c84fc95a9cd2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 74c5b5be-8c07-4870-ad96-3bd00809d0ec (zpool), kind: Internal } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
452025-01-08T18:19:56.048ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
462025-01-08T18:19:56.048ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
472025-01-08T18:19:56.048ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
482025-01-08T18:19:56.048ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
492025-01-08T18:19:56.048ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_1.vdev
502025-01-08T18:19:56.049ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
512025-01-08T18:19:56.049ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: e4d4e46e-f335-4351-a367-c84fc95a9cd2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 74c5b5be-8c07-4870-ad96-3bd00809d0ec (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
522025-01-08T18:19:56.049ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
532025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
542025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
552025-01-08T18:19:56.049ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_2.vdev
562025-01-08T18:19:56.049ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
572025-01-08T18:19:56.049ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: e4d4e46e-f335-4351-a367-c84fc95a9cd2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 74c5b5be-8c07-4870-ad96-3bd00809d0ec (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
582025-01-08T18:19:56.049ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
592025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
602025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
612025-01-08T18:19:56.049ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_3.vdev
622025-01-08T18:19:56.049ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
632025-01-08T18:19:56.049ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: e4d4e46e-f335-4351-a367-c84fc95a9cd2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 74c5b5be-8c07-4870-ad96-3bd00809d0ec (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
642025-01-08T18:19:56.049ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
652025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
662025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
672025-01-08T18:19:56.049ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_4.vdev
682025-01-08T18:19:56.049ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: e4d4e46e-f335-4351-a367-c84fc95a9cd2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 74c5b5be-8c07-4870-ad96-3bd00809d0ec (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
692025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
702025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
712025-01-08T18:19:56.049ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
722025-01-08T18:19:56.049ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
732025-01-08T18:19:56.049ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_5.vdev
742025-01-08T18:19:56.049ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: e4d4e46e-f335-4351-a367-c84fc95a9cd2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 74c5b5be-8c07-4870-ad96-3bd00809d0ec (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
752025-01-08T18:19:56.049ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
762025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
772025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
782025-01-08T18:19:56.049ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
792025-01-08T18:19:56.049ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_6.vdev
802025-01-08T18:19:56.049ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: e4d4e46e-f335-4351-a367-c84fc95a9cd2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 74c5b5be-8c07-4870-ad96-3bd00809d0ec (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
812025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
822025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
832025-01-08T18:19:56.049ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
842025-01-08T18:19:56.049ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
852025-01-08T18:19:56.049ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_7.vdev
862025-01-08T18:19:56.049ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: e4d4e46e-f335-4351-a367-c84fc95a9cd2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 74c5b5be-8c07-4870-ad96-3bd00809d0ec (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
872025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
882025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
892025-01-08T18:19:56.049ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
902025-01-08T18:19:56.049ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
912025-01-08T18:19:56.049ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_8.vdev
922025-01-08T18:19:56.049ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
932025-01-08T18:19:56.049ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
942025-01-08T18:19:56.049ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: e4d4e46e-f335-4351-a367-c84fc95a9cd2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 74c5b5be-8c07-4870-ad96-3bd00809d0ec (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
952025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
962025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
972025-01-08T18:19:56.049ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:97
982025-01-08T18:19:56.049ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-agent/src/long_running_tasks.rs:99
992025-01-08T18:19:56.049ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
1002025-01-08T18:19:56.049ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: e4d4e46e-f335-4351-a367-c84fc95a9cd2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 74c5b5be-8c07-4870-ad96-3bd00809d0ec (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
1012025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
1022025-01-08T18:19:56.049ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1032025-01-08T18:19:56.049ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:211
1042025-01-08T18:19:56.049ZINFOSledAgent (bootstore): No ledger in /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1052025-01-08T18:19:56.049ZINFOSledAgent (bootstore): No ledger in /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1062025-01-08T18:19:56.049ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-buskin-i86pc
1072025-01-08T18:19:56.049ZINFOSledAgent (bootstore): No ledger in /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1082025-01-08T18:19:56.049ZINFOSledAgent (bootstore): No ledger in /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-buskin-i86pc
1092025-01-08T18:19:56.049ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-buskin-i86pc
1102025-01-08T18:19:56.049ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:216
1112025-01-08T18:19:56.049ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:231
1122025-01-08T18:19:56.060ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:748
1132025-01-08T18:19:56.060ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 4169, tv_nsec: 931290921 } time_to_next_cleanup = 599.999937369s
1142025-01-08T18:19:56.060ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3973
1152025-01-08T18:19:56.060ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
1162025-01-08T18:19:56.063ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/sled-agent-request.json file = common/src/ledger.rs:188
1172025-01-08T18:19:56.063ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/sled-agent-request.json file = common/src/ledger.rs:188
1182025-01-08T18:19:56.064ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80
1192025-01-08T18:19:56.065ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:51 local_addr = [fdb0:18c0:4d0d:9fb2::1]:12346
1202025-01-08T18:19:56.066ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.1.164, last: 10.151.1.173, }, ), ], external_dns_ips: [ 10.151.1.164, 10.151.1.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.1.100, vlan_id: None, rib_priority: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.1.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: Some( None, ), bgp_peers: [], autoneg: false, lldp: None, tx_eq: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, } file = sled-agent/src/rack_setup/service.rs:1203
1212025-01-08T18:19:56.066ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1222025-01-08T18:19:56.066ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
1232025-01-08T18:19:56.066ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
1242025-01-08T18:19:56.066ZINFOSledAgent (RSS): No ledger in /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/rss-started.marker file = common/src/ledger.rs:188
1252025-01-08T18:19:56.066ZINFOSledAgent (RSS): No ledger in /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/rss-started.marker file = common/src/ledger.rs:188
1262025-01-08T18:19:56.066ZINFOSledAgent (RSS): No ledger in /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1272025-01-08T18:19:56.066ZINFOSledAgent (RSS): No ledger in /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1282025-01-08T18:19:56.066ZINFOSledAgent (RSS): RSS not previously run. Creating plans. file = sled-agent/src/rack_setup/service.rs:1252
1292025-01-08T18:19:56.066ZINFOSledAgent (RSS): Writing ledger to /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/.rss-started.marker.tmp file = common/src/ledger.rs:199
1302025-01-08T18:19:56.066ZINFOSledAgent (RSS): Writing ledger to /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/.rss-started.marker.tmp file = common/src/ledger.rs:199
1312025-01-08T18:19:56.066ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:1284
1322025-01-08T18:19:56.066ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:66
1332025-01-08T18:19:56.066ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:9fb2::1 file = sled-agent/src/rack_setup/plan/sled.rs:39
1342025-01-08T18:19:56.066ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:1316
1352025-01-08T18:19:56.066ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-buskin-i86pc
1362025-01-08T18:19:56.066ZINFOSledAgent (bootstore): Writing ledger to /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
1372025-01-08T18:19:56.067ZINFOSledAgent (bootstore): Writing ledger to /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
1382025-01-08T18:19:56.067ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-buskin-i86pc
1392025-01-08T18:19:56.067ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:232 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 (sled), rack_id: dc521fbd-1e78-4c20-a78c-fc16c751ec23, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } target_sled = [fdb0:18c0:4d0d:9fb2::1]:12346
1402025-01-08T18:19:56.067ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:4270
1412025-01-08T18:19:56.067ZINFOSledAgent (ServiceManager): Starting switch zone file = sled-agent/src/services.rs:4664
1422025-01-08T18:19:56.072ZINFOSledAgent (SledAgentSprocketsClient): Loaded keys and certs BootstrapAgentClient = [fdb0:18c0:4d0d:9fb2::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/client.rs:50
1432025-01-08T18:19:56.078ZINFOSledAgent (SledAgentSprocketsClient): Certificate chain verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:9fb2::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1442025-01-08T18:19:56.078ZINFOSledAgent (SledAgentSprocketsClient): Signature verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:9fb2::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1452025-01-08T18:19:56.083ZINFOSledAgent (SledAgentSprocketsServer): Certificate chain verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1462025-01-08T18:19:56.083ZINFOSledAgent (SledAgentSprocketsServer): Signature verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1472025-01-08T18:19:56.083ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:74 remote_addr = [fdb0:18c0:4d0d:9fb2::1]:38310
1482025-01-08T18:19:56.083ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 (sled), rack_id: dc521fbd-1e78-4c20-a78c-fc16c751ec23, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } file = sled-agent/src/bootstrap/server.rs:362
1492025-01-08T18:19:56.083ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:377
1502025-01-08T18:19:56.083ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:44
1512025-01-08T18:19:56.083ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1522025-01-08T18:19:56.083ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:607
1532025-01-08T18:19:56.083ZINFOSledAgent (StorageManager): Loading storage config file = sled-storage/src/manager.rs:813
1542025-01-08T18:19:56.083ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:101::, width: 64 }
1552025-01-08T18:19:56.083ZINFOSledAgent (StorageManager): No ledger in /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1562025-01-08T18:19:56.083ZINFOSledAgent (StorageManager): No ledger in /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1572025-01-08T18:19:56.083ZINFOSledAgent (StorageManager): No ledger of physical disks exists file = sled-storage/src/manager.rs:802
1582025-01-08T18:19:56.083ZINFOSledAgent (StorageManager): KeyManager ready, but no ledger detected file = sled-storage/src/manager.rs:830
1592025-01-08T18:19:56.083ZINFOSledAgent (StorageManager): automatically managing disks file = sled-storage/src/manager.rs:596
1602025-01-08T18:19:56.083ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
1612025-01-08T18:19:56.083ZINFOSledAgent (StorageManager): Successfully synchronized disks without error file = sled-storage/src/manager.rs:701 result = DisksManagementResult { status: [] }
1622025-01-08T18:19:56.098ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
1632025-01-08T18:19:56.119ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:325
1642025-01-08T18:19:56.137ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:363
1652025-01-08T18:19:56.148ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:412 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
1662025-01-08T18:19:56.148ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
1672025-01-08T18:19:56.148ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:424 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
1682025-01-08T18:19:56.148ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1692025-01-08T18:19:56.151ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 136.919322ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "4d7b05dc-27a0-4faf-a75b-450dfe800d4b", "content-length": "124", "date": "Wed, 08 Jan 2025 18:19:56 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "4d7b05dc-27a0-4faf-a75b-450dfe800d4b" } file = clients/ddm-admin-client/src/lib.rs:177
1702025-01-08T18:19:56.161ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_e4d4e46e-f335-4351-a367-c84fc95a9cd2/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1712025-01-08T18:19:56.185ZINFOSledAgent: successfully created encrypted zvol: name="oxi_e4d4e46e-f335-4351-a367-c84fc95a9cd2/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1722025-01-08T18:19:56.185ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_e4d4e46e-f335-4351-a367-c84fc95a9cd2/swap" file = sled-agent/src/swap_device.rs:96
1732025-01-08T18:19:56.191ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:439 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
1742025-01-08T18:19:56.191ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:129
1752025-01-08T18:19:56.191ZINFOSledAgent (BackingFs): Ensuring dataset oxi_e4d4e46e-f335-4351-a367-c84fc95a9cd2/backing/fmd file = sled-agent/src/backing_fs.rs:139
1762025-01-08T18:19:56.267ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:174
1772025-01-08T18:19:56.289ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1782025-01-08T18:19:56.289ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 111.658465ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "822f0956-f501-4611-81ac-f2b76f985ead", "content-length": "124", "date": "Wed, 08 Jan 2025 18:19:56 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "822f0956-f501-4611-81ac-f2b76f985ead" } file = clients/ddm-admin-client/src/lib.rs:177
1792025-01-08T18:19:56.402ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1802025-01-08T18:19:56.403ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 202.614744ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "46032be0-98af-4ab6-9a56-ad31d5148c6c", "content-length": "124", "date": "Wed, 08 Jan 2025 18:19:56 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "46032be0-98af-4ab6-9a56-ad31d5148c6c" } file = clients/ddm-admin-client/src/lib.rs:177
1812025-01-08T18:19:56.486ZINFOSledAgent (BackingFs): Mounting oxi_e4d4e46e-f335-4351-a367-c84fc95a9cd2/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:182
1822025-01-08T18:19:56.498ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:190
1832025-01-08T18:19:56.499ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:190
1842025-01-08T18:19:56.499ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:190
1852025-01-08T18:19:56.499ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:196
1862025-01-08T18:19:56.607ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1872025-01-08T18:19:56.607ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 118.6835ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "7c624219-e345-4de5-a988-3cf30434061e", "content-length": "124", "date": "Wed, 08 Jan 2025 18:19:56 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "7c624219-e345-4de5-a988-3cf30434061e" } file = clients/ddm-admin-client/src/lib.rs:177
1882025-01-08T18:19:56.727ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1892025-01-08T18:19:56.727ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 153.610817ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "78854467-e5e0-4d33-b4c2-dd207fd40807", "content-length": "124", "date": "Wed, 08 Jan 2025 18:19:56 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "78854467-e5e0-4d33-b4c2-dd207fd40807" } file = clients/ddm-admin-client/src/lib.rs:177
1902025-01-08T18:19:56.882ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1912025-01-08T18:19:56.883ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 254.953287ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "fa67e258-efc6-4e09-b26e-459ff91c5dcf", "content-length": "124", "date": "Wed, 08 Jan 2025 18:19:56 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "fa67e258-efc6-4e09-b26e-459ff91c5dcf" } file = clients/ddm-admin-client/src/lib.rs:177
1922025-01-08T18:19:57.139ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1932025-01-08T18:19:57.140ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 149.403474ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "57a72c96-be01-45f1-bc79-790f6afd228d", "content-length": "124", "date": "Wed, 08 Jan 2025 18:19:57 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "57a72c96-be01-45f1-bc79-790f6afd228d" } file = clients/ddm-admin-client/src/lib.rs:177
1942025-01-08T18:19:57.236ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:93 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
1952025-01-08T18:19:57.242ZINFOSledAgent (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:33327 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
1962025-01-08T18:19:57.242ZINFOSledAgent (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
1972025-01-08T18:19:57.242ZINFOSledAgent (producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::1]:33327 file = oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
1982025-01-08T18:19:57.243ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 147.931688ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
1992025-01-08T18:19:57.244ZINFOSledAgent (VmmReservoirManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/vmm_reservoir.rs:267 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
2002025-01-08T18:19:57.244ZINFOSledAgent (VmmReservoirManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:275 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
2012025-01-08T18:19:57.244ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:932 underlay_address = fd00:1122:3344:101::1
2022025-01-08T18:19:57.244ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
2032025-01-08T18:19:57.290ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
2042025-01-08T18:19:57.393ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 738.501634ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
2052025-01-08T18:19:58.133ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 1.410928046s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
2062025-01-08T18:19:59.546ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 1.07741784s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
2072025-01-08T18:19:59.910ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2694
2082025-01-08T18:19:59.910ZINFOSledAgent (ServiceManager): Setting up lldpd service file = sled-agent/src/services.rs:2939
2092025-01-08T18:19:59.910ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2564
2102025-01-08T18:19:59.910ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2611
2112025-01-08T18:19:59.910ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:3020
2122025-01-08T18:19:59.910ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:3065
2132025-01-08T18:19:59.912ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2864
2142025-01-08T18:19:59.914ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2605
2152025-01-08T18:19:59.914ZINFOSledAgent (ServiceManager): Profile for oxz_switch: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/mgs"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12225'/> <propval type="astring" name="id" value='914e2acd-bd80-48ad-b32b-a58e12480009'/> <propval type="astring" name="rack_id" value='dc521fbd-1e78-4c20-a78c-fc16c751ec23'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/wicketd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12226'/> <propval type="astring" name="artifact-address" value='[fdb0:18c0:4d0d:9fb2::2]:12227'/> <propval type="astring" name="baseboard-file" value='/opt/oxide/baseboard.json'/> <propval type="astring" name="mgs-address" value='[::1]:12225'/> <propval type="astring" name="nexus-proxy-address" value='[::]:12229'/> <propval type="astring" name="rack-subnet" value='fd00:1122:3344::'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/switch_zone_setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="gz_local_link_addr" value='fe80::8:20ff:fe8b:e121'/> <property type="astring" name="link_local_links"> <astring_list> <value_node value="tfportrear0_0" /> <value_node value="oxBootstrap0" /> </astring_list> </property> <propval type="astring" name="bootstrap_addr" value='fdb0:18c0:4d0d:9fb2::2'/> <propval type="astring" name="bootstrap_vnic" value='oxBootstrap0'/> <propval type="astring" name="baseboard_info" value='{"type":"pc","identifier":"buskin","model":"i86pc"}'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/dendrite"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="rack_id" value='dc521fbd-1e78-4c20-a78c-fc16c751ec23'/> <propval type="astring" name="sled_id" value='7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0'/> <propval type="astring" name="sled_model" value='i86pc'/> <propval type="astring" name="sled_serial" value='buskin'/> <propval type="astring" name="sled_revision" value='0'/> <propval type="astring" name="address" value='[::1]:12224'/> <propval type="astring" name="front_ports" value='1'/> <propval type="astring" name="rear_ports" value='1'/> <propval type="astring" name="port_config" value='/opt/oxide/dendrite/misc/softnpu_single_sled_config.toml'/> <propval type="astring" name="mgmt" value='uds'/> <propval type="astring" name="uds_path" value='/opt/softnpu/stuff'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/tfport"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="host" value='[::1]'/> <propval type="astring" name="port" value='12224'/> <propval type="astring" name="flags" value='--sync-only'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/lldpd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="board_rev" value='softnpu_front_1_rear_1'/> <propval type="astring" name="scrimlet_id" value='buskin'/> <propval type="astring" name="scrimlet_model" value='i86pc'/> <propval type="astring" name="address" value='[::1]:12230'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/pumpkind"> </service> <service version="1" type="service" name="oxide/mgd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_uuid" value='7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0'/> <propval type="astring" name="rack_uuid" value='dc521fbd-1e78-4c20-a78c-fc16c751ec23'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/mg-ddm"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="mode" value='transit'/> <propval type="astring" name="dendrite" value='true'/> <propval type="astring" name="sled_uuid" value='7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0'/> <propval type="astring" name="rack_uuid" value='dc521fbd-1e78-4c20-a78c-fc16c751ec23'/> <propval type="astring" name="interfaces" value='tfportrear0_0/ll'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/uplink"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
2162025-01-08T18:19:59.914ZINFOSledAgent (ServiceManager): Booting oxz_switch zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_switch
2172025-01-08T18:20:00.172ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 31.963367ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2182025-01-08T18:20:00.398ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 142.406498ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2192025-01-08T18:20:00.581ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 269.722461ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2202025-01-08T18:20:00.625ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 3.459378986s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
2212025-01-08T18:20:00.916ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 369.007471ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2222025-01-08T18:20:01.308ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 848.430734ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2232025-01-08T18:20:02.378ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 899.507066ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2242025-01-08T18:20:03.480ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 575.288953ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2252025-01-08T18:20:04.172ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 747.700038ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2262025-01-08T18:20:04.173ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 9.208957899s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
2272025-01-08T18:20:05.111ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 953.2738ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2282025-01-08T18:20:06.377ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0) file = sled-agent/src/nexus.rs:527 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
2292025-01-08T18:20:06.408ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
2302025-01-08T18:20:06.408ZINFOSledAgent (dropshot (Repo Depot)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:12348 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
2312025-01-08T18:20:06.409ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:12345
2322025-01-08T18:20:06.409ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:432
2332025-01-08T18:20:06.409ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
2342025-01-08T18:20:06.409ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2352025-01-08T18:20:06.409ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2362025-01-08T18:20:06.410ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:138
2372025-01-08T18:20:06.410ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56 }, infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [PortConfigV2 { routes: [RouteConfig { destination: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), nexthop: 10.151.1.100, vlan_id: None, rib_priority: None }], addresses: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: Some(None), bgp_peers: [], autoneg: false, lldp: None, tx_eq: None }], bgp: [], bfd: [] })))
2382025-01-08T18:20:06.410ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3973
2392025-01-08T18:20:06.410ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:81 remote_addr = [fdb0:18c0:4d0d:9fb2::1]:38310
2402025-01-08T18:20:06.641ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:117 peer = [fdb0:18c0:4d0d:9fb2::1]:12346
2412025-01-08T18:20:06.641ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:252 target_sled = [fdb0:18c0:4d0d:9fb2::1]:12346
2422025-01-08T18:20:06.647ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:4294 new = [fd00:1122:3344:101::2, ::1] old = [::1]
2432025-01-08T18:20:07.335ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 18250165013537883834 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
2442025-01-08T18:20:07.335ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 2727085820797177907 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
2452025-01-08T18:20:07.581ZINFOSledAgent (VmmReservoirManager): Finished setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:279 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
2462025-01-08T18:20:07.582ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50390
2472025-01-08T18:20:07.583ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
2482025-01-08T18:20:07.583ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
2492025-01-08T18:20:07.583ZINFOSledAgent (ServiceManager): No ledger in /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-zones.json file = common/src/ledger.rs:188
2502025-01-08T18:20:07.583ZINFOSledAgent (ServiceManager): No ledger in /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-zones.json file = common/src/ledger.rs:188
2512025-01-08T18:20:07.583ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 443 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:50390 req_id = e34b4d56-ce34-4304-a067-7c088f873043 response_code = 200 uri = /inventory
2522025-01-08T18:20:07.583ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0) file = sled-agent/src/probe_manager.rs:231 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
2532025-01-08T18:20:07.600ZINFOSledAgent (ServiceManager): refreshed zone-network-setup service with new configuration file = sled-agent/src/services.rs:4343
2542025-01-08T18:20:07.600ZINFOSledAgent (ServiceManager): configuring dendrite service file = sled-agent/src/services.rs:4401
2552025-01-08T18:20:07.628ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:51096
2562025-01-08T18:20:07.629ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:51096 req_id = ab7682a7-0a83-4e10-95b1-ac3e6ac534f3 response_code = 200 uri = /sled-role
2572025-01-08T18:20:07.674ZINFOSledAgent (RSS): attempting to set up sled's storage: OmicronPhysicalDisksConfig { generation: Generation(2), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 3b516190-2afe-4e65-be71-40cd8e508952 (physical_disk), pool_id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: ef03275c-3698-45f9-89f2-be39395863c5 (physical_disk), pool_id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: a6cc2e20-a8cf-41c6-9f7d-92b982bc554e (physical_disk), pool_id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: d1cfc3a6-0467-4345-83dd-7a2ceb284729 (physical_disk), pool_id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 74dcbe74-5355-432d-a331-b2c988192539 (physical_disk), pool_id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 4a8ef84b-7026-4127-835e-d30edb67376f (physical_disk), pool_id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 5ce061fb-1683-43f1-9459-538f23d779f1 (physical_disk), pool_id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: dfea744c-9664-4ab1-8201-30695b5f16c2 (physical_disk), pool_id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: af618717-8d3c-47a5-bc20-8b483ee3d657 (physical_disk), pool_id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool) }] } file = sled-agent/src/rack_setup/service.rs:385 sled_address = [fd00:1122:3344:101::1]:12345
2582025-01-08T18:20:07.674ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:37689
2592025-01-08T18:20:07.674ZINFOSledAgent: physical disks ensure file = sled-agent/src/sled_agent.rs:893 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
2602025-01-08T18:20:07.674ZINFOSledAgent (StorageManager): Received OmicronPhysicalDisksEnsure { config: OmicronPhysicalDisksConfig { generation: Generation(2), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 3b516190-2afe-4e65-be71-40cd8e508952 (physical_disk), pool_id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: ef03275c-3698-45f9-89f2-be39395863c5 (physical_disk), pool_id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: a6cc2e20-a8cf-41c6-9f7d-92b982bc554e (physical_disk), pool_id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: d1cfc3a6-0467-4345-83dd-7a2ceb284729 (physical_disk), pool_id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 74dcbe74-5355-432d-a331-b2c988192539 (physical_disk), pool_id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 4a8ef84b-7026-4127-835e-d30edb67376f (physical_disk), pool_id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 5ce061fb-1683-43f1-9459-538f23d779f1 (physical_disk), pool_id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: dfea744c-9664-4ab1-8201-30695b5f16c2 (physical_disk), pool_id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: af618717-8d3c-47a5-bc20-8b483ee3d657 (physical_disk), pool_id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool) }] }, tx: ... } file = sled-storage/src/manager.rs:607
2612025-01-08T18:20:07.675ZINFOSledAgent (StorageManager): No ledger in /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2622025-01-08T18:20:07.675ZINFOSledAgent (StorageManager): No ledger in /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2632025-01-08T18:20:07.675ZINFOSledAgent (StorageManager): No previously-stored 'requested disks', creating new ledger file = sled-storage/src/manager.rs:1298 request = omicron_physical_disks_ensure
2642025-01-08T18:20:07.675ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
2652025-01-08T18:20:07.675ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:342
2662025-01-08T18:20:07.675ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2672025-01-08T18:20:07.675ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } path = /scratch/u2_0.vdev
2682025-01-08T18:20:07.685ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_0.vdev file = sled-hardware/src/disk.rs:367
2692025-01-08T18:20:07.685ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool)
2702025-01-08T18:20:07.748ZINFOSledAgent (ServiceManager): refreshed dendrite service with new configuration file = sled-agent/src/services.rs:4467
2712025-01-08T18:20:07.748ZINFOSledAgent (ServiceManager): configuring lldp service file = sled-agent/src/services.rs:4494
2722025-01-08T18:20:07.788ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }
2732025-01-08T18:20:07.791ZINFOSledAgent (ServiceManager): refreshed lldpd service with new configuration file = sled-agent/src/services.rs:4507
2742025-01-08T18:20:07.791ZINFOSledAgent (ServiceManager): configuring MGS service file = sled-agent/src/services.rs:4353
2752025-01-08T18:20:07.798ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:239
2762025-01-08T18:20:07.798ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:241
2772025-01-08T18:20:07.798ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:245
2782025-01-08T18:20:07.798ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:247
2792025-01-08T18:20:07.798ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2802025-01-08T18:20:07.798ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2812025-01-08T18:20:07.798ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
2822025-01-08T18:20:07.837ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2832025-01-08T18:20:07.847ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/zone file = sled-storage/src/dataset.rs:302
2842025-01-08T18:20:07.984ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f file = sled-storage/src/dataset.rs:388
2852025-01-08T18:20:07.992ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:440
2862025-01-08T18:20:07.992ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:357
2872025-01-08T18:20:07.992ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:342
2882025-01-08T18:20:07.992ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2892025-01-08T18:20:07.992ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } path = /scratch/u2_1.vdev
2902025-01-08T18:20:08.002ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_1.vdev file = sled-hardware/src/disk.rs:367
2912025-01-08T18:20:08.002ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool)
2922025-01-08T18:20:08.097ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }
2932025-01-08T18:20:08.106ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:239
2942025-01-08T18:20:08.106ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:241
2952025-01-08T18:20:08.106ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:245
2962025-01-08T18:20:08.106ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:247
2972025-01-08T18:20:08.106ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2982025-01-08T18:20:08.106ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2992025-01-08T18:20:08.106ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
3002025-01-08T18:20:08.140ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3012025-01-08T18:20:08.147ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_0ac8ff68-15ea-4477-a206-5a7619973e85/crypt/zone file = sled-storage/src/dataset.rs:302
3022025-01-08T18:20:08.287ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_0ac8ff68-15ea-4477-a206-5a7619973e85 file = sled-storage/src/dataset.rs:388
3032025-01-08T18:20:08.295ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:440
3042025-01-08T18:20:08.295ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:357
3052025-01-08T18:20:08.295ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:342
3062025-01-08T18:20:08.295ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3072025-01-08T18:20:08.295ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } path = /scratch/u2_2.vdev
3082025-01-08T18:20:08.304ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_2.vdev file = sled-hardware/src/disk.rs:367
3092025-01-08T18:20:08.304ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool)
3102025-01-08T18:20:08.398ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }
3112025-01-08T18:20:08.406ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:239
3122025-01-08T18:20:08.406ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:241
3132025-01-08T18:20:08.406ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:245
3142025-01-08T18:20:08.406ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:247
3152025-01-08T18:20:08.406ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3162025-01-08T18:20:08.406ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3172025-01-08T18:20:08.406ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
3182025-01-08T18:20:08.442ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3192025-01-08T18:20:08.450ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_bf665b71-2255-48d6-b1fd-c863b4c8e1af/crypt/zone file = sled-storage/src/dataset.rs:302
3202025-01-08T18:20:08.583ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_bf665b71-2255-48d6-b1fd-c863b4c8e1af file = sled-storage/src/dataset.rs:388
3212025-01-08T18:20:08.586ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0) file = sled-agent/src/probe_manager.rs:231 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
3222025-01-08T18:20:08.591ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:440
3232025-01-08T18:20:08.591ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:357
3242025-01-08T18:20:08.591ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:342
3252025-01-08T18:20:08.591ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3262025-01-08T18:20:08.591ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } path = /scratch/u2_3.vdev
3272025-01-08T18:20:08.600ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_3.vdev file = sled-hardware/src/disk.rs:367
3282025-01-08T18:20:08.600ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool)
3292025-01-08T18:20:08.692ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }
3302025-01-08T18:20:08.700ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:239
3312025-01-08T18:20:08.700ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:241
3322025-01-08T18:20:08.700ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:245
3332025-01-08T18:20:08.700ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:247
3342025-01-08T18:20:08.700ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3352025-01-08T18:20:08.700ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3362025-01-08T18:20:08.700ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
3372025-01-08T18:20:08.733ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3382025-01-08T18:20:08.741ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_4f399814-e4ff-4480-aa92-c4c936f90fd0/crypt/zone file = sled-storage/src/dataset.rs:302
3392025-01-08T18:20:08.882ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_4f399814-e4ff-4480-aa92-c4c936f90fd0 file = sled-storage/src/dataset.rs:388
3402025-01-08T18:20:08.890ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:440
3412025-01-08T18:20:08.890ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:357
3422025-01-08T18:20:08.890ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:342
3432025-01-08T18:20:08.890ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3442025-01-08T18:20:08.890ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } path = /scratch/u2_4.vdev
3452025-01-08T18:20:08.900ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_4.vdev file = sled-hardware/src/disk.rs:367
3462025-01-08T18:20:08.900ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool)
3472025-01-08T18:20:08.996ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }
3482025-01-08T18:20:09.005ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:239
3492025-01-08T18:20:09.005ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:241
3502025-01-08T18:20:09.005ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:245
3512025-01-08T18:20:09.005ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:247
3522025-01-08T18:20:09.005ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3532025-01-08T18:20:09.005ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3542025-01-08T18:20:09.005ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
3552025-01-08T18:20:09.040ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3562025-01-08T18:20:09.048ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_c1f40097-34cd-4b0e-8a1d-a4c6931740f6/crypt/zone file = sled-storage/src/dataset.rs:302
3572025-01-08T18:20:09.191ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_c1f40097-34cd-4b0e-8a1d-a4c6931740f6 file = sled-storage/src/dataset.rs:388
3582025-01-08T18:20:09.199ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:440
3592025-01-08T18:20:09.199ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:357
3602025-01-08T18:20:09.199ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:342
3612025-01-08T18:20:09.199ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3622025-01-08T18:20:09.199ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } path = /scratch/u2_5.vdev
3632025-01-08T18:20:09.209ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_5.vdev file = sled-hardware/src/disk.rs:367
3642025-01-08T18:20:09.209ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = efc412bd-0718-4381-8d37-e5a204c8c3de (zpool)
3652025-01-08T18:20:09.301ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }
3662025-01-08T18:20:09.309ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:239
3672025-01-08T18:20:09.309ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:241
3682025-01-08T18:20:09.309ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:245
3692025-01-08T18:20:09.309ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:247
3702025-01-08T18:20:09.309ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3712025-01-08T18:20:09.309ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3722025-01-08T18:20:09.309ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
3732025-01-08T18:20:09.346ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3742025-01-08T18:20:09.354ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_efc412bd-0718-4381-8d37-e5a204c8c3de/crypt/zone file = sled-storage/src/dataset.rs:302
3752025-01-08T18:20:09.498ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_efc412bd-0718-4381-8d37-e5a204c8c3de file = sled-storage/src/dataset.rs:388
3762025-01-08T18:20:09.506ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:440
3772025-01-08T18:20:09.506ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:357
3782025-01-08T18:20:09.506ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:342
3792025-01-08T18:20:09.506ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3802025-01-08T18:20:09.506ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } path = /scratch/u2_6.vdev
3812025-01-08T18:20:09.516ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_6.vdev file = sled-hardware/src/disk.rs:367
3822025-01-08T18:20:09.516ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool)
3832025-01-08T18:20:09.614ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }
3842025-01-08T18:20:09.622ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:239
3852025-01-08T18:20:09.622ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:241
3862025-01-08T18:20:09.622ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:245
3872025-01-08T18:20:09.622ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:247
3882025-01-08T18:20:09.622ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3892025-01-08T18:20:09.622ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3902025-01-08T18:20:09.622ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
3912025-01-08T18:20:09.626ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0) file = sled-agent/src/probe_manager.rs:231 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
3922025-01-08T18:20:09.658ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3932025-01-08T18:20:09.666ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_91e62592-6bd8-45a2-a311-bfef0a5977f4/crypt/zone file = sled-storage/src/dataset.rs:302
3942025-01-08T18:20:09.806ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_91e62592-6bd8-45a2-a311-bfef0a5977f4 file = sled-storage/src/dataset.rs:388
3952025-01-08T18:20:09.814ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:440
3962025-01-08T18:20:09.814ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:357
3972025-01-08T18:20:09.814ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:342
3982025-01-08T18:20:09.814ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3992025-01-08T18:20:09.814ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } path = /scratch/u2_7.vdev
4002025-01-08T18:20:09.824ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_7.vdev file = sled-hardware/src/disk.rs:367
4012025-01-08T18:20:09.824ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool)
4022025-01-08T18:20:09.845ZINFOSledAgent (ServiceManager): refreshed MGS service with new configuration file = sled-agent/src/services.rs:4395
4032025-01-08T18:20:09.845ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:4474 rack_subnet = fd00:1122:3344::
4042025-01-08T18:20:09.925ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }
4052025-01-08T18:20:09.933ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:239
4062025-01-08T18:20:09.933ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:241
4072025-01-08T18:20:09.933ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:245
4082025-01-08T18:20:09.933ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:247
4092025-01-08T18:20:09.933ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4102025-01-08T18:20:09.933ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4112025-01-08T18:20:09.933ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
4122025-01-08T18:20:09.969ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4132025-01-08T18:20:09.978ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_93dafc63-6f00-46e6-a292-a8413d960ce4/crypt/zone file = sled-storage/src/dataset.rs:302
4142025-01-08T18:20:10.121ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_93dafc63-6f00-46e6-a292-a8413d960ce4 file = sled-storage/src/dataset.rs:388
4152025-01-08T18:20:10.129ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:440
4162025-01-08T18:20:10.129ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:357
4172025-01-08T18:20:10.129ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:342
4182025-01-08T18:20:10.129ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4192025-01-08T18:20:10.129ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } path = /scratch/u2_8.vdev
4202025-01-08T18:20:10.139ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_8.vdev file = sled-hardware/src/disk.rs:367
4212025-01-08T18:20:10.139ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 36139540-6634-4f0c-9c90-f57efa2432be (zpool)
4222025-01-08T18:20:10.255ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External }
4232025-01-08T18:20:10.263ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:239
4242025-01-08T18:20:10.264ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:241
4252025-01-08T18:20:10.264ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:245
4262025-01-08T18:20:10.264ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:247
4272025-01-08T18:20:10.264ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4282025-01-08T18:20:10.264ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4292025-01-08T18:20:10.264ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
4302025-01-08T18:20:10.302ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4312025-01-08T18:20:10.310ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_36139540-6634-4f0c-9c90-f57efa2432be/crypt/zone file = sled-storage/src/dataset.rs:302
4322025-01-08T18:20:10.449ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_36139540-6634-4f0c-9c90-f57efa2432be file = sled-storage/src/dataset.rs:388
4332025-01-08T18:20:10.457ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:440
4342025-01-08T18:20:10.457ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:357
4352025-01-08T18:20:10.457ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4362025-01-08T18:20:10.457ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(2), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: e4d4e46e-f335-4351-a367-c84fc95a9cd2 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 74c5b5be-8c07-4870-ad96-3bd00809d0ec (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
4372025-01-08T18:20:10.457ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
4382025-01-08T18:20:10.457ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
4392025-01-08T18:20:10.457ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4402025-01-08T18:20:10.458ZINFOSledAgent: physical disks ensure: Updated storage file = sled-agent/src/sled_agent.rs:897 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
4412025-01-08T18:20:10.458ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
4422025-01-08T18:20:10.458ZINFOSledAgent: physical disks ensure: Propagating new generation of disks file = sled-agent/src/sled_agent.rs:923 generation = Generation(2) sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
4432025-01-08T18:20:10.458ZINFOSledAgent: physical disks ensure: Updated storage monitor file = sled-agent/src/sled_agent.rs:928 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
4442025-01-08T18:20:10.458ZINFOSledAgent: physical disks ensure: Updated zone bundler file = sled-agent/src/sled_agent.rs:933 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
4452025-01-08T18:20:10.458ZINFOSledAgent (ProbeManager): use_only_these_disks: Processing new request file = sled-agent/src/probe_manager.rs:122 gen = Generation(2) sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
4462025-01-08T18:20:10.458ZINFOSledAgent: physical disks ensure: Updated probes file = sled-agent/src/sled_agent.rs:939 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
4472025-01-08T18:20:10.458ZINFOSledAgent (InstanceManager): use_only_these_disks: Processing new request file = sled-agent/src/instance_manager.rs:799 gen = Generation(2)
4482025-01-08T18:20:10.458ZINFOSledAgent: physical disks ensure: Updated instances file = sled-agent/src/sled_agent.rs:944 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
4492025-01-08T18:20:10.458ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2783686 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:37689 req_id = 3315695f-0107-4cbd-9442-c0cf980f8897 response_code = 200 uri = /omicron-physical-disks
4502025-01-08T18:20:10.490ZINFOSledAgent (RSS): attempting to set up sled's datasets: DatasetsConfig { generation: Generation(1), datasets: {07679c90-ef85-4893-96f5-123e9a1ad5b9 (dataset): DatasetConfig { id: 07679c90-ef85-4893-96f5-123e9a1ad5b9 (dataset), name: DatasetName { pool_name: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 092e51a8-0c83-4cb7-9a2c-669ad82c93c9 (dataset): DatasetConfig { id: 092e51a8-0c83-4cb7-9a2c-669ad82c93c9 (dataset), name: DatasetName { pool_name: ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0a4cc2b0-2967-44bb-bc36-f7548c61cdcc (dataset): DatasetConfig { id: 0a4cc2b0-2967-44bb-bc36-f7548c61cdcc (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0ae1132f-6770-44b1-9d43-4a132d5570a2 (dataset): DatasetConfig { id: 0ae1132f-6770-44b1-9d43-4a132d5570a2 (dataset), name: DatasetName { pool_name: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 0beead28-854f-476e-aaaa-bf598a8b8ec8 (dataset): DatasetConfig { id: 0beead28-854f-476e-aaaa-bf598a8b8ec8 (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 17780736-4096-4b8a-9201-f7a48ee0f152 (dataset): DatasetConfig { id: 17780736-4096-4b8a-9201-f7a48ee0f152 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1c49c06a-1251-4011-aad4-0da7a4f40e1e (dataset): DatasetConfig { id: 1c49c06a-1251-4011-aad4-0da7a4f40e1e (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1c57817d-8567-4ad5-bbb2-087c519e6f13 (dataset): DatasetConfig { id: 1c57817d-8567-4ad5-bbb2-087c519e6f13 (dataset), name: DatasetName { pool_name: ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2229cdd9-9ba3-4a5e-b8ee-6f1eb7424e75 (dataset): DatasetConfig { id: 2229cdd9-9ba3-4a5e-b8ee-6f1eb7424e75 (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 26548d45-3e3b-4124-96ae-056682748435 (dataset): DatasetConfig { id: 26548d45-3e3b-4124-96ae-056682748435 (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2c32426a-c547-4877-a0a5-ac447fcafc8b (dataset): DatasetConfig { id: 2c32426a-c547-4877-a0a5-ac447fcafc8b (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2fff5fa1-2b22-4880-a123-57a910532f1a (dataset): DatasetConfig { id: 2fff5fa1-2b22-4880-a123-57a910532f1a (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 347d519e-f09b-4194-86f7-7428120a7c60 (dataset): DatasetConfig { id: 347d519e-f09b-4194-86f7-7428120a7c60 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 35d69a66-2c7f-4cc6-8332-649051fadf90 (dataset): DatasetConfig { id: 35d69a66-2c7f-4cc6-8332-649051fadf90 (dataset), name: DatasetName { pool_name: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3981e7e5-012c-4e7b-888d-480e8882389d (dataset): DatasetConfig { id: 3981e7e5-012c-4e7b-888d-480e8882389d (dataset), name: DatasetName { pool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 3cd43eb3-7da9-4dd6-82e4-a4843cec367a (dataset): DatasetConfig { id: 3cd43eb3-7da9-4dd6-82e4-a4843cec367a (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4245fac5-b765-41fe-be07-d3b4d04c4537 (dataset): DatasetConfig { id: 4245fac5-b765-41fe-be07-d3b4d04c4537 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4487008f-dc8e-4ad8-b201-22a3ae3baa77 (dataset): DatasetConfig { id: 4487008f-dc8e-4ad8-b201-22a3ae3baa77 (dataset), name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 44a95c9e-5fa7-4539-976c-ad93c5a77b3f (dataset): DatasetConfig { id: 44a95c9e-5fa7-4539-976c-ad93c5a77b3f (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 45ca3dee-479b-44fb-ba9c-c02b1ae169fb (dataset): DatasetConfig { id: 45ca3dee-479b-44fb-ba9c-c02b1ae169fb (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 582e4825-bbb9-4047-a2ca-289c4867f440 (dataset): DatasetConfig { id: 582e4825-bbb9-4047-a2ca-289c4867f440 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 5b7c3500-d554-4e3a-bc98-98eaa3e865c6 (dataset): DatasetConfig { id: 5b7c3500-d554-4e3a-bc98-98eaa3e865c6 (dataset), name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6257dcb0-712b-4d8c-8e4f-82f780281399 (dataset): DatasetConfig { id: 6257dcb0-712b-4d8c-8e4f-82f780281399 (dataset), name: DatasetName { pool_name: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 62f3c4dd-9a5d-4c76-9000-a2fcbb4dbe5e (dataset): DatasetConfig { id: 62f3c4dd-9a5d-4c76-9000-a2fcbb4dbe5e (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 62fd612c-7a95-4cdf-9aba-6cf8f5a32311 (dataset): DatasetConfig { id: 62fd612c-7a95-4cdf-9aba-6cf8f5a32311 (dataset), name: DatasetName { pool_name: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 63d8c39b-8b94-457b-b3cd-206a4c112476 (dataset): DatasetConfig { id: 63d8c39b-8b94-457b-b3cd-206a4c112476 (dataset), name: DatasetName { pool_name: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 681a9fb0-59fe-4b1e-8397-b1a8ec06de09 (dataset): DatasetConfig { id: 681a9fb0-59fe-4b1e-8397-b1a8ec06de09 (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6ba3475c-ba6c-4400-aea9-74c4bf718568 (dataset): DatasetConfig { id: 6ba3475c-ba6c-4400-aea9-74c4bf718568 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 713c2acf-b29f-4ad6-974e-91dc8790c733 (dataset): DatasetConfig { id: 713c2acf-b29f-4ad6-974e-91dc8790c733 (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 78a8300a-f1e1-4c5b-b297-3a62aaae9046 (dataset): DatasetConfig { id: 78a8300a-f1e1-4c5b-b297-3a62aaae9046 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 7cd1882f-a0bb-43dc-8fa9-e96631a990ea (dataset): DatasetConfig { id: 7cd1882f-a0bb-43dc-8fa9-e96631a990ea (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8854e5b0-672d-40c5-9243-e10a7d30df83 (dataset): DatasetConfig { id: 8854e5b0-672d-40c5-9243-e10a7d30df83 (dataset), name: DatasetName { pool_name: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8896597c-e4d4-440e-b1bb-974462040136 (dataset): DatasetConfig { id: 8896597c-e4d4-440e-b1bb-974462040136 (dataset), name: DatasetName { pool_name: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 95b8dd7a-1840-4ba6-9452-30d3fc1b65ce (dataset): DatasetConfig { id: 95b8dd7a-1840-4ba6-9452-30d3fc1b65ce (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Clickhouse }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 963ae7c5-484e-40be-8517-5755cb36d70b (dataset): DatasetConfig { id: 963ae7c5-484e-40be-8517-5755cb36d70b (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9852b436-406b-42da-af94-93f6fd8d8ca6 (dataset): DatasetConfig { id: 9852b436-406b-42da-af94-93f6fd8d8ca6 (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: TransientZone { name: "oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 99ad0023-083d-4e7e-a740-9e566a8666d2 (dataset): DatasetConfig { id: 99ad0023-083d-4e7e-a740-9e566a8666d2 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9c21a55a-63a1-4c51-a48e-86d29b6a08b8 (dataset): DatasetConfig { id: 9c21a55a-63a1-4c51-a48e-86d29b6a08b8 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9d81b8b1-3b31-4bf6-a564-4a0986a5340b (dataset): DatasetConfig { id: 9d81b8b1-3b31-4bf6-a564-4a0986a5340b (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a34530d8-9fe2-48ae-829f-e51e8d249f6c (dataset): DatasetConfig { id: a34530d8-9fe2-48ae-829f-e51e8d249f6c (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a591eb99-8bb3-456e-8dc8-2028b44fd789 (dataset): DatasetConfig { id: a591eb99-8bb3-456e-8dc8-2028b44fd789 (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ad04d36b-8076-4127-ad16-22fcd554c838 (dataset): DatasetConfig { id: ad04d36b-8076-4127-ad16-22fcd554c838 (dataset), name: DatasetName { pool_name: ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b2e086ec-3876-4f98-a9f1-ccab6777e132 (dataset): DatasetConfig { id: b2e086ec-3876-4f98-a9f1-ccab6777e132 (dataset), name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b6a9feb9-4858-4a6c-b0be-e582ec5eefcd (dataset): DatasetConfig { id: b6a9feb9-4858-4a6c-b0be-e582ec5eefcd (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, b6bc3f81-295c-4450-82c4-b9e790a8c327 (dataset): DatasetConfig { id: b6bc3f81-295c-4450-82c4-b9e790a8c327 (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b95feae1-e8fd-4082-8459-7e471372800b (dataset): DatasetConfig { id: b95feae1-e8fd-4082-8459-7e471372800b (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, bbb0435c-a651-475a-99d9-529e4b4614a7 (dataset): DatasetConfig { id: bbb0435c-a651-475a-99d9-529e4b4614a7 (dataset), name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c38ff0cc-44f5-41b6-a47a-626a926f040a (dataset): DatasetConfig { id: c38ff0cc-44f5-41b6-a47a-626a926f040a (dataset), name: DatasetName { pool_name: ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, caa337b0-ac5e-4be1-8bb5-80f053a45445 (dataset): DatasetConfig { id: caa337b0-ac5e-4be1-8bb5-80f053a45445 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ce0d0aa3-d00f-448f-8050-6b1c0c4c4c2b (dataset): DatasetConfig { id: ce0d0aa3-d00f-448f-8050-6b1c0c4c4c2b (dataset), name: DatasetName { pool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d2c47ebe-5131-4f5a-887b-89f8cb8180a6 (dataset): DatasetConfig { id: d2c47ebe-5131-4f5a-887b-89f8cb8180a6 (dataset), name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, d5c0d042-2665-458d-8893-4b457347b4a0 (dataset): DatasetConfig { id: d5c0d042-2665-458d-8893-4b457347b4a0 (dataset), name: DatasetName { pool_name: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }, kind: TransientZone { name: "oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d5de69fb-aee9-4ca6-81e7-c63715d8b24d (dataset): DatasetConfig { id: d5de69fb-aee9-4ca6-81e7-c63715d8b24d (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d69d28d8-55b8-4f2e-b198-57f94a05a2aa (dataset): DatasetConfig { id: d69d28d8-55b8-4f2e-b198-57f94a05a2aa (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d7961443-14d0-4a60-a47c-42a2c854aa59 (dataset): DatasetConfig { id: d7961443-14d0-4a60-a47c-42a2c854aa59 (dataset), name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, dbc20397-7ba7-4720-bec9-247aa6886ee9 (dataset): DatasetConfig { id: dbc20397-7ba7-4720-bec9-247aa6886ee9 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, dd40988f-0cfc-4afe-9f52-4b5a5c40cb65 (dataset): DatasetConfig { id: dd40988f-0cfc-4afe-9f52-4b5a5c40cb65 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, df7350d3-98f1-4b74-9d27-8e0db7404a0e (dataset): DatasetConfig { id: df7350d3-98f1-4b74-9d27-8e0db7404a0e (dataset), name: DatasetName { pool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e6770bb5-e9e8-4653-a3dd-e735e33e86a6 (dataset): DatasetConfig { id: e6770bb5-e9e8-4653-a3dd-e735e33e86a6 (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e7387b2a-56ad-42b1-a648-12d27872673b (dataset): DatasetConfig { id: e7387b2a-56ad-42b1-a648-12d27872673b (dataset), name: DatasetName { pool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, kind: TransientZone { name: "oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ed9805b7-c1b7-42bb-bb6b-ef8605a7b5b0 (dataset): DatasetConfig { id: ed9805b7-c1b7-42bb-bb6b-ef8605a7b5b0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, eec7c537-27c7-4ae7-a19c-b7a7a483022b (dataset): DatasetConfig { id: eec7c537-27c7-4ae7-a19c-b7a7a483022b (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, f91b460d-8aad-411f-8321-d552ef7caac0 (dataset): DatasetConfig { id: f91b460d-8aad-411f-8321-d552ef7caac0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f96dc87a-0516-46b5-9f63-6763cb8cba0f (dataset): DatasetConfig { id: f96dc87a-0516-46b5-9f63-6763cb8cba0f (dataset), name: DatasetName { pool_name: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, fdfc388d-4b07-455d-97ff-6a6865ec0c33 (dataset): DatasetConfig { id: fdfc388d-4b07-455d-97ff-6a6865ec0c33 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, fff8ece3-710e-4462-b538-01d66871ffc0 (dataset): DatasetConfig { id: fff8ece3-710e-4462-b538-01d66871ffc0 (dataset), name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }} } file = sled-agent/src/rack_setup/service.rs:460 sled_address = [fd00:1122:3344:101::1]:12345
4512025-01-08T18:20:10.490ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:40092
4522025-01-08T18:20:10.490ZINFOSledAgent: datasets ensure file = sled-agent/src/sled_agent.rs:859 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
4532025-01-08T18:20:10.490ZINFOSledAgent (StorageManager): Received DatasetsEnsure { config: DatasetsConfig { generation: Generation(1), datasets: {07679c90-ef85-4893-96f5-123e9a1ad5b9 (dataset): DatasetConfig { id: 07679c90-ef85-4893-96f5-123e9a1ad5b9 (dataset), name: DatasetName { pool_name: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 092e51a8-0c83-4cb7-9a2c-669ad82c93c9 (dataset): DatasetConfig { id: 092e51a8-0c83-4cb7-9a2c-669ad82c93c9 (dataset), name: DatasetName { pool_name: ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0a4cc2b0-2967-44bb-bc36-f7548c61cdcc (dataset): DatasetConfig { id: 0a4cc2b0-2967-44bb-bc36-f7548c61cdcc (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0ae1132f-6770-44b1-9d43-4a132d5570a2 (dataset): DatasetConfig { id: 0ae1132f-6770-44b1-9d43-4a132d5570a2 (dataset), name: DatasetName { pool_name: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 0beead28-854f-476e-aaaa-bf598a8b8ec8 (dataset): DatasetConfig { id: 0beead28-854f-476e-aaaa-bf598a8b8ec8 (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 17780736-4096-4b8a-9201-f7a48ee0f152 (dataset): DatasetConfig { id: 17780736-4096-4b8a-9201-f7a48ee0f152 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1c49c06a-1251-4011-aad4-0da7a4f40e1e (dataset): DatasetConfig { id: 1c49c06a-1251-4011-aad4-0da7a4f40e1e (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1c57817d-8567-4ad5-bbb2-087c519e6f13 (dataset): DatasetConfig { id: 1c57817d-8567-4ad5-bbb2-087c519e6f13 (dataset), name: DatasetName { pool_name: ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2229cdd9-9ba3-4a5e-b8ee-6f1eb7424e75 (dataset): DatasetConfig { id: 2229cdd9-9ba3-4a5e-b8ee-6f1eb7424e75 (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 26548d45-3e3b-4124-96ae-056682748435 (dataset): DatasetConfig { id: 26548d45-3e3b-4124-96ae-056682748435 (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2c32426a-c547-4877-a0a5-ac447fcafc8b (dataset): DatasetConfig { id: 2c32426a-c547-4877-a0a5-ac447fcafc8b (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2fff5fa1-2b22-4880-a123-57a910532f1a (dataset): DatasetConfig { id: 2fff5fa1-2b22-4880-a123-57a910532f1a (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 347d519e-f09b-4194-86f7-7428120a7c60 (dataset): DatasetConfig { id: 347d519e-f09b-4194-86f7-7428120a7c60 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 35d69a66-2c7f-4cc6-8332-649051fadf90 (dataset): DatasetConfig { id: 35d69a66-2c7f-4cc6-8332-649051fadf90 (dataset), name: DatasetName { pool_name: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3981e7e5-012c-4e7b-888d-480e8882389d (dataset): DatasetConfig { id: 3981e7e5-012c-4e7b-888d-480e8882389d (dataset), name: DatasetName { pool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 3cd43eb3-7da9-4dd6-82e4-a4843cec367a (dataset): DatasetConfig { id: 3cd43eb3-7da9-4dd6-82e4-a4843cec367a (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4245fac5-b765-41fe-be07-d3b4d04c4537 (dataset): DatasetConfig { id: 4245fac5-b765-41fe-be07-d3b4d04c4537 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4487008f-dc8e-4ad8-b201-22a3ae3baa77 (dataset): DatasetConfig { id: 4487008f-dc8e-4ad8-b201-22a3ae3baa77 (dataset), name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 44a95c9e-5fa7-4539-976c-ad93c5a77b3f (dataset): DatasetConfig { id: 44a95c9e-5fa7-4539-976c-ad93c5a77b3f (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 45ca3dee-479b-44fb-ba9c-c02b1ae169fb (dataset): DatasetConfig { id: 45ca3dee-479b-44fb-ba9c-c02b1ae169fb (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 582e4825-bbb9-4047-a2ca-289c4867f440 (dataset): DatasetConfig { id: 582e4825-bbb9-4047-a2ca-289c4867f440 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 5b7c3500-d554-4e3a-bc98-98eaa3e865c6 (dataset): DatasetConfig { id: 5b7c3500-d554-4e3a-bc98-98eaa3e865c6 (dataset), name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6257dcb0-712b-4d8c-8e4f-82f780281399 (dataset): DatasetConfig { id: 6257dcb0-712b-4d8c-8e4f-82f780281399 (dataset), name: DatasetName { pool_name: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 62f3c4dd-9a5d-4c76-9000-a2fcbb4dbe5e (dataset): DatasetConfig { id: 62f3c4dd-9a5d-4c76-9000-a2fcbb4dbe5e (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 62fd612c-7a95-4cdf-9aba-6cf8f5a32311 (dataset): DatasetConfig { id: 62fd612c-7a95-4cdf-9aba-6cf8f5a32311 (dataset), name: DatasetName { pool_name: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 63d8c39b-8b94-457b-b3cd-206a4c112476 (dataset): DatasetConfig { id: 63d8c39b-8b94-457b-b3cd-206a4c112476 (dataset), name: DatasetName { pool_name: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 681a9fb0-59fe-4b1e-8397-b1a8ec06de09 (dataset): DatasetConfig { id: 681a9fb0-59fe-4b1e-8397-b1a8ec06de09 (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6ba3475c-ba6c-4400-aea9-74c4bf718568 (dataset): DatasetConfig { id: 6ba3475c-ba6c-4400-aea9-74c4bf718568 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 713c2acf-b29f-4ad6-974e-91dc8790c733 (dataset): DatasetConfig { id: 713c2acf-b29f-4ad6-974e-91dc8790c733 (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 78a8300a-f1e1-4c5b-b297-3a62aaae9046 (dataset): DatasetConfig { id: 78a8300a-f1e1-4c5b-b297-3a62aaae9046 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 7cd1882f-a0bb-43dc-8fa9-e96631a990ea (dataset): DatasetConfig { id: 7cd1882f-a0bb-43dc-8fa9-e96631a990ea (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8854e5b0-672d-40c5-9243-e10a7d30df83 (dataset): DatasetConfig { id: 8854e5b0-672d-40c5-9243-e10a7d30df83 (dataset), name: DatasetName { pool_name: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8896597c-e4d4-440e-b1bb-974462040136 (dataset): DatasetConfig { id: 8896597c-e4d4-440e-b1bb-974462040136 (dataset), name: DatasetName { pool_name: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 95b8dd7a-1840-4ba6-9452-30d3fc1b65ce (dataset): DatasetConfig { id: 95b8dd7a-1840-4ba6-9452-30d3fc1b65ce (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Clickhouse }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 963ae7c5-484e-40be-8517-5755cb36d70b (dataset): DatasetConfig { id: 963ae7c5-484e-40be-8517-5755cb36d70b (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9852b436-406b-42da-af94-93f6fd8d8ca6 (dataset): DatasetConfig { id: 9852b436-406b-42da-af94-93f6fd8d8ca6 (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: TransientZone { name: "oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 99ad0023-083d-4e7e-a740-9e566a8666d2 (dataset): DatasetConfig { id: 99ad0023-083d-4e7e-a740-9e566a8666d2 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9c21a55a-63a1-4c51-a48e-86d29b6a08b8 (dataset): DatasetConfig { id: 9c21a55a-63a1-4c51-a48e-86d29b6a08b8 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9d81b8b1-3b31-4bf6-a564-4a0986a5340b (dataset): DatasetConfig { id: 9d81b8b1-3b31-4bf6-a564-4a0986a5340b (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a34530d8-9fe2-48ae-829f-e51e8d249f6c (dataset): DatasetConfig { id: a34530d8-9fe2-48ae-829f-e51e8d249f6c (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a591eb99-8bb3-456e-8dc8-2028b44fd789 (dataset): DatasetConfig { id: a591eb99-8bb3-456e-8dc8-2028b44fd789 (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ad04d36b-8076-4127-ad16-22fcd554c838 (dataset): DatasetConfig { id: ad04d36b-8076-4127-ad16-22fcd554c838 (dataset), name: DatasetName { pool_name: ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b2e086ec-3876-4f98-a9f1-ccab6777e132 (dataset): DatasetConfig { id: b2e086ec-3876-4f98-a9f1-ccab6777e132 (dataset), name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b6a9feb9-4858-4a6c-b0be-e582ec5eefcd (dataset): DatasetConfig { id: b6a9feb9-4858-4a6c-b0be-e582ec5eefcd (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, b6bc3f81-295c-4450-82c4-b9e790a8c327 (dataset): DatasetConfig { id: b6bc3f81-295c-4450-82c4-b9e790a8c327 (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b95feae1-e8fd-4082-8459-7e471372800b (dataset): DatasetConfig { id: b95feae1-e8fd-4082-8459-7e471372800b (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, bbb0435c-a651-475a-99d9-529e4b4614a7 (dataset): DatasetConfig { id: bbb0435c-a651-475a-99d9-529e4b4614a7 (dataset), name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c38ff0cc-44f5-41b6-a47a-626a926f040a (dataset): DatasetConfig { id: c38ff0cc-44f5-41b6-a47a-626a926f040a (dataset), name: DatasetName { pool_name: ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, caa337b0-ac5e-4be1-8bb5-80f053a45445 (dataset): DatasetConfig { id: caa337b0-ac5e-4be1-8bb5-80f053a45445 (dataset), name: DatasetName { pool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ce0d0aa3-d00f-448f-8050-6b1c0c4c4c2b (dataset): DatasetConfig { id: ce0d0aa3-d00f-448f-8050-6b1c0c4c4c2b (dataset), name: DatasetName { pool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d2c47ebe-5131-4f5a-887b-89f8cb8180a6 (dataset): DatasetConfig { id: d2c47ebe-5131-4f5a-887b-89f8cb8180a6 (dataset), name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, d5c0d042-2665-458d-8893-4b457347b4a0 (dataset): DatasetConfig { id: d5c0d042-2665-458d-8893-4b457347b4a0 (dataset), name: DatasetName { pool_name: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }, kind: TransientZone { name: "oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d5de69fb-aee9-4ca6-81e7-c63715d8b24d (dataset): DatasetConfig { id: d5de69fb-aee9-4ca6-81e7-c63715d8b24d (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d69d28d8-55b8-4f2e-b198-57f94a05a2aa (dataset): DatasetConfig { id: d69d28d8-55b8-4f2e-b198-57f94a05a2aa (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d7961443-14d0-4a60-a47c-42a2c854aa59 (dataset): DatasetConfig { id: d7961443-14d0-4a60-a47c-42a2c854aa59 (dataset), name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, dbc20397-7ba7-4720-bec9-247aa6886ee9 (dataset): DatasetConfig { id: dbc20397-7ba7-4720-bec9-247aa6886ee9 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, dd40988f-0cfc-4afe-9f52-4b5a5c40cb65 (dataset): DatasetConfig { id: dd40988f-0cfc-4afe-9f52-4b5a5c40cb65 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, df7350d3-98f1-4b74-9d27-8e0db7404a0e (dataset): DatasetConfig { id: df7350d3-98f1-4b74-9d27-8e0db7404a0e (dataset), name: DatasetName { pool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e6770bb5-e9e8-4653-a3dd-e735e33e86a6 (dataset): DatasetConfig { id: e6770bb5-e9e8-4653-a3dd-e735e33e86a6 (dataset), name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e7387b2a-56ad-42b1-a648-12d27872673b (dataset): DatasetConfig { id: e7387b2a-56ad-42b1-a648-12d27872673b (dataset), name: DatasetName { pool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, kind: TransientZone { name: "oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ed9805b7-c1b7-42bb-bb6b-ef8605a7b5b0 (dataset): DatasetConfig { id: ed9805b7-c1b7-42bb-bb6b-ef8605a7b5b0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, eec7c537-27c7-4ae7-a19c-b7a7a483022b (dataset): DatasetConfig { id: eec7c537-27c7-4ae7-a19c-b7a7a483022b (dataset), name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, f91b460d-8aad-411f-8321-d552ef7caac0 (dataset): DatasetConfig { id: f91b460d-8aad-411f-8321-d552ef7caac0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f96dc87a-0516-46b5-9f63-6763cb8cba0f (dataset): DatasetConfig { id: f96dc87a-0516-46b5-9f63-6763cb8cba0f (dataset), name: DatasetName { pool_name: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, fdfc388d-4b07-455d-97ff-6a6865ec0c33 (dataset): DatasetConfig { id: fdfc388d-4b07-455d-97ff-6a6865ec0c33 (dataset), name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, fff8ece3-710e-4462-b538-01d66871ffc0 (dataset): DatasetConfig { id: fff8ece3-710e-4462-b538-01d66871ffc0 (dataset), name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }} }, tx: ... } file = sled-storage/src/manager.rs:607
4542025-01-08T18:20:10.490ZINFOSledAgent (StorageManager): No ledger in /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-datasets.json file = common/src/ledger.rs:188 request = datasets_ensure
4552025-01-08T18:20:10.490ZINFOSledAgent (StorageManager): No ledger in /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-datasets.json file = common/src/ledger.rs:188 request = datasets_ensure
4562025-01-08T18:20:10.490ZINFOSledAgent (StorageManager): No previously-stored 'requested datasets', creating new ledger file = sled-storage/src/manager.rs:909 request = datasets_ensure
4572025-01-08T18:20:10.505ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4582025-01-08T18:20:10.505ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4592025-01-08T18:20:10.505ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_91e62592-6bd8-45a2-a311-bfef0a5977f4/crucible" } file = sled-storage/src/manager.rs:1431
4602025-01-08T18:20:10.552ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4612025-01-08T18:20:10.552ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4622025-01-08T18:20:10.552ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/36139540-6634-4f0c-9c90-f57efa2432be/crypt/zone/oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b"), full_name: "oxp_36139540-6634-4f0c-9c90-f57efa2432be/crypt/zone/oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b" } file = sled-storage/src/manager.rs:1431
4632025-01-08T18:20:10.611ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4642025-01-08T18:20:10.611ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4652025-01-08T18:20:10.611ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/zone/oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f"), full_name: "oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/zone/oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f" } file = sled-storage/src/manager.rs:1431
4662025-01-08T18:20:10.629ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0) file = sled-agent/src/probe_manager.rs:231 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
4672025-01-08T18:20:10.673ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4682025-01-08T18:20:10.673ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
4692025-01-08T18:20:10.673ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/91e62592-6bd8-45a2-a311-bfef0a5977f4/crypt/debug"), full_name: "oxp_91e62592-6bd8-45a2-a311-bfef0a5977f4/crypt/debug" } file = sled-storage/src/manager.rs:1431
4702025-01-08T18:20:10.705ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4712025-01-08T18:20:10.705ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4722025-01-08T18:20:10.705ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crucible" } file = sled-storage/src/manager.rs:1431
4732025-01-08T18:20:10.755ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4742025-01-08T18:20:10.755ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4752025-01-08T18:20:10.755ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_93dafc63-6f00-46e6-a292-a8413d960ce4/crucible" } file = sled-storage/src/manager.rs:1431
4762025-01-08T18:20:10.802ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4772025-01-08T18:20:10.802ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4782025-01-08T18:20:10.802ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_bf665b71-2255-48d6-b1fd-c863b4c8e1af/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1431
4792025-01-08T18:20:10.852ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4802025-01-08T18:20:10.852ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
4812025-01-08T18:20:10.852ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/36139540-6634-4f0c-9c90-f57efa2432be/crypt/zone"), full_name: "oxp_36139540-6634-4f0c-9c90-f57efa2432be/crypt/zone" } file = sled-storage/src/manager.rs:1431
4822025-01-08T18:20:10.880ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4832025-01-08T18:20:10.880ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4842025-01-08T18:20:10.880ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/bf665b71-2255-48d6-b1fd-c863b4c8e1af/crypt/zone/oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766"), full_name: "oxp_bf665b71-2255-48d6-b1fd-c863b4c8e1af/crypt/zone/oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766" } file = sled-storage/src/manager.rs:1431
4852025-01-08T18:20:10.940ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4862025-01-08T18:20:10.940ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4872025-01-08T18:20:10.940ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1431
4882025-01-08T18:20:10.989ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4892025-01-08T18:20:10.989ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
4902025-01-08T18:20:10.989ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0ac8ff68-15ea-4477-a206-5a7619973e85/crypt/zone"), full_name: "oxp_0ac8ff68-15ea-4477-a206-5a7619973e85/crypt/zone" } file = sled-storage/src/manager.rs:1431
4912025-01-08T18:20:11.018ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4922025-01-08T18:20:11.018ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
4932025-01-08T18:20:11.018ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4f399814-e4ff-4480-aa92-c4c936f90fd0/crypt/zone"), full_name: "oxp_4f399814-e4ff-4480-aa92-c4c936f90fd0/crypt/zone" } file = sled-storage/src/manager.rs:1431
4942025-01-08T18:20:11.047ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4952025-01-08T18:20:11.047ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4962025-01-08T18:20:11.047ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4f399814-e4ff-4480-aa92-c4c936f90fd0/crypt/zone/oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd"), full_name: "oxp_4f399814-e4ff-4480-aa92-c4c936f90fd0/crypt/zone/oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd" } file = sled-storage/src/manager.rs:1431
4972025-01-08T18:20:11.107ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4982025-01-08T18:20:11.107ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
4992025-01-08T18:20:11.107ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/efc412bd-0718-4381-8d37-e5a204c8c3de/crypt/zone"), full_name: "oxp_efc412bd-0718-4381-8d37-e5a204c8c3de/crypt/zone" } file = sled-storage/src/manager.rs:1431
5002025-01-08T18:20:11.136ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5012025-01-08T18:20:11.136ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5022025-01-08T18:20:11.136ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/93dafc63-6f00-46e6-a292-a8413d960ce4/crypt/debug"), full_name: "oxp_93dafc63-6f00-46e6-a292-a8413d960ce4/crypt/debug" } file = sled-storage/src/manager.rs:1431
5032025-01-08T18:20:11.165ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5042025-01-08T18:20:11.165ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5052025-01-08T18:20:11.165ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/zone/oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77"), full_name: "oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/zone/oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77" } file = sled-storage/src/manager.rs:1431
5062025-01-08T18:20:11.225ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5072025-01-08T18:20:11.225ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5082025-01-08T18:20:11.225ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4f399814-e4ff-4480-aa92-c4c936f90fd0/crypt/zone/oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501"), full_name: "oxp_4f399814-e4ff-4480-aa92-c4c936f90fd0/crypt/zone/oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501" } file = sled-storage/src/manager.rs:1431
5092025-01-08T18:20:11.286ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5102025-01-08T18:20:11.286ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5112025-01-08T18:20:11.286ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c1f40097-34cd-4b0e-8a1d-a4c6931740f6/crypt/zone/oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8"), full_name: "oxp_c1f40097-34cd-4b0e-8a1d-a4c6931740f6/crypt/zone/oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8" } file = sled-storage/src/manager.rs:1431
5122025-01-08T18:20:11.348ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5132025-01-08T18:20:11.348ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5142025-01-08T18:20:11.348ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/bf665b71-2255-48d6-b1fd-c863b4c8e1af/crypt/zone/oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3"), full_name: "oxp_bf665b71-2255-48d6-b1fd-c863b4c8e1af/crypt/zone/oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3" } file = sled-storage/src/manager.rs:1431
5152025-01-08T18:20:11.408ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5162025-01-08T18:20:11.409ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5172025-01-08T18:20:11.409ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0ac8ff68-15ea-4477-a206-5a7619973e85/crypt/zone/oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4"), full_name: "oxp_0ac8ff68-15ea-4477-a206-5a7619973e85/crypt/zone/oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4" } file = sled-storage/src/manager.rs:1431
5182025-01-08T18:20:11.468ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5192025-01-08T18:20:11.468ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5202025-01-08T18:20:11.468ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0ac8ff68-15ea-4477-a206-5a7619973e85/crypt/debug"), full_name: "oxp_0ac8ff68-15ea-4477-a206-5a7619973e85/crypt/debug" } file = sled-storage/src/manager.rs:1431
5212025-01-08T18:20:11.497ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5222025-01-08T18:20:11.497ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5232025-01-08T18:20:11.497ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_c1f40097-34cd-4b0e-8a1d-a4c6931740f6/crucible" } file = sled-storage/src/manager.rs:1431
5242025-01-08T18:20:11.545ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5252025-01-08T18:20:11.545ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5262025-01-08T18:20:11.545ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/efc412bd-0718-4381-8d37-e5a204c8c3de/crypt/debug"), full_name: "oxp_efc412bd-0718-4381-8d37-e5a204c8c3de/crypt/debug" } file = sled-storage/src/manager.rs:1431
5272025-01-08T18:20:11.575ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5282025-01-08T18:20:11.575ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5292025-01-08T18:20:11.575ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0ac8ff68-15ea-4477-a206-5a7619973e85/crypt/zone/oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f"), full_name: "oxp_0ac8ff68-15ea-4477-a206-5a7619973e85/crypt/zone/oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f" } file = sled-storage/src/manager.rs:1431
5302025-01-08T18:20:11.631ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0) file = sled-agent/src/probe_manager.rs:231 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
5312025-01-08T18:20:11.633ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5322025-01-08T18:20:11.633ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5332025-01-08T18:20:11.633ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/91e62592-6bd8-45a2-a311-bfef0a5977f4/crypt/zone"), full_name: "oxp_91e62592-6bd8-45a2-a311-bfef0a5977f4/crypt/zone" } file = sled-storage/src/manager.rs:1431
5342025-01-08T18:20:11.662ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5352025-01-08T18:20:11.662ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5362025-01-08T18:20:11.663ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/efc412bd-0718-4381-8d37-e5a204c8c3de/crypt/zone/oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a"), full_name: "oxp_efc412bd-0718-4381-8d37-e5a204c8c3de/crypt/zone/oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a" } file = sled-storage/src/manager.rs:1431
5372025-01-08T18:20:11.722ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5382025-01-08T18:20:11.722ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5392025-01-08T18:20:11.722ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/internal_dns" } file = sled-storage/src/manager.rs:1431
5402025-01-08T18:20:11.771ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5412025-01-08T18:20:11.771ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5422025-01-08T18:20:11.771ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/93dafc63-6f00-46e6-a292-a8413d960ce4/crypt/zone/oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e"), full_name: "oxp_93dafc63-6f00-46e6-a292-a8413d960ce4/crypt/zone/oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e" } file = sled-storage/src/manager.rs:1431
5432025-01-08T18:20:11.833ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5442025-01-08T18:20:11.833ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5452025-01-08T18:20:11.833ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/external_dns" } file = sled-storage/src/manager.rs:1431
5462025-01-08T18:20:11.871ZINFOSledAgent (ServiceManager): refreshed wicketd service with new configuration file = sled-agent/src/services.rs:4485
5472025-01-08T18:20:11.871ZINFOSledAgent (ServiceManager): configuring mgd service file = sled-agent/src/services.rs:4527
5482025-01-08T18:20:11.884ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5492025-01-08T18:20:11.884ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5502025-01-08T18:20:11.884ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0ac8ff68-15ea-4477-a206-5a7619973e85/crypt/zone/oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3"), full_name: "oxp_0ac8ff68-15ea-4477-a206-5a7619973e85/crypt/zone/oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3" } file = sled-storage/src/manager.rs:1431
5512025-01-08T18:20:11.947ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5522025-01-08T18:20:11.947ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5532025-01-08T18:20:11.947ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/zone"), full_name: "oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/zone" } file = sled-storage/src/manager.rs:1431
5542025-01-08T18:20:11.976ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5552025-01-08T18:20:11.976ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5562025-01-08T18:20:11.976ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_efc412bd-0718-4381-8d37-e5a204c8c3de/crucible" } file = sled-storage/src/manager.rs:1431
5572025-01-08T18:20:12.027ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5582025-01-08T18:20:12.027ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5592025-01-08T18:20:12.027ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/91e62592-6bd8-45a2-a311-bfef0a5977f4/crypt/zone/oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd"), full_name: "oxp_91e62592-6bd8-45a2-a311-bfef0a5977f4/crypt/zone/oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd" } file = sled-storage/src/manager.rs:1431
5602025-01-08T18:20:12.086ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5612025-01-08T18:20:12.086ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5622025-01-08T18:20:12.086ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/clickhouse" } file = sled-storage/src/manager.rs:1431
5632025-01-08T18:20:12.136ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5642025-01-08T18:20:12.136ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5652025-01-08T18:20:12.136ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/zone/oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb"), full_name: "oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/zone/oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb" } file = sled-storage/src/manager.rs:1431
5662025-01-08T18:20:12.194ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5672025-01-08T18:20:12.194ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5682025-01-08T18:20:12.194ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/zone/oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f"), full_name: "oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/zone/oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f" } file = sled-storage/src/manager.rs:1431
5692025-01-08T18:20:12.254ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5702025-01-08T18:20:12.254ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5712025-01-08T18:20:12.254ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0ac8ff68-15ea-4477-a206-5a7619973e85/crypt/zone/oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58"), full_name: "oxp_0ac8ff68-15ea-4477-a206-5a7619973e85/crypt/zone/oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58" } file = sled-storage/src/manager.rs:1431
5722025-01-08T18:20:12.314ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5732025-01-08T18:20:12.315ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5742025-01-08T18:20:12.315ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_0ac8ff68-15ea-4477-a206-5a7619973e85/crypt/external_dns" } file = sled-storage/src/manager.rs:1431
5752025-01-08T18:20:12.364ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5762025-01-08T18:20:12.364ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5772025-01-08T18:20:12.364ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_4f399814-e4ff-4480-aa92-c4c936f90fd0/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1431
5782025-01-08T18:20:12.413ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5792025-01-08T18:20:12.413ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5802025-01-08T18:20:12.413ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/bf665b71-2255-48d6-b1fd-c863b4c8e1af/crypt/zone/oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1"), full_name: "oxp_bf665b71-2255-48d6-b1fd-c863b4c8e1af/crypt/zone/oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1" } file = sled-storage/src/manager.rs:1431
5812025-01-08T18:20:12.472ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5822025-01-08T18:20:12.472ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5832025-01-08T18:20:12.472ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_bf665b71-2255-48d6-b1fd-c863b4c8e1af/crucible" } file = sled-storage/src/manager.rs:1431
5842025-01-08T18:20:12.520ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5852025-01-08T18:20:12.520ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5862025-01-08T18:20:12.520ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_36139540-6634-4f0c-9c90-f57efa2432be/crucible" } file = sled-storage/src/manager.rs:1431
5872025-01-08T18:20:12.568ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5882025-01-08T18:20:12.568ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5892025-01-08T18:20:12.568ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c1f40097-34cd-4b0e-8a1d-a4c6931740f6/crypt/zone/oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63"), full_name: "oxp_c1f40097-34cd-4b0e-8a1d-a4c6931740f6/crypt/zone/oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63" } file = sled-storage/src/manager.rs:1431
5902025-01-08T18:20:12.627ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5912025-01-08T18:20:12.628ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5922025-01-08T18:20:12.628ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/bf665b71-2255-48d6-b1fd-c863b4c8e1af/crypt/debug"), full_name: "oxp_bf665b71-2255-48d6-b1fd-c863b4c8e1af/crypt/debug" } file = sled-storage/src/manager.rs:1431
5932025-01-08T18:20:12.634ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0) file = sled-agent/src/probe_manager.rs:231 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
5942025-01-08T18:20:12.657ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5952025-01-08T18:20:12.657ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5962025-01-08T18:20:12.657ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/bf665b71-2255-48d6-b1fd-c863b4c8e1af/crypt/zone/oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa"), full_name: "oxp_bf665b71-2255-48d6-b1fd-c863b4c8e1af/crypt/zone/oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa" } file = sled-storage/src/manager.rs:1431
5972025-01-08T18:20:12.718ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5982025-01-08T18:20:12.718ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5992025-01-08T18:20:12.718ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_0ac8ff68-15ea-4477-a206-5a7619973e85/crucible" } file = sled-storage/src/manager.rs:1431
6002025-01-08T18:20:12.765ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6012025-01-08T18:20:12.765ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6022025-01-08T18:20:12.765ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c1f40097-34cd-4b0e-8a1d-a4c6931740f6/crypt/zone/oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795"), full_name: "oxp_c1f40097-34cd-4b0e-8a1d-a4c6931740f6/crypt/zone/oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795" } file = sled-storage/src/manager.rs:1431
6032025-01-08T18:20:12.824ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6042025-01-08T18:20:12.824ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
6052025-01-08T18:20:12.824ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/36139540-6634-4f0c-9c90-f57efa2432be/crypt/debug"), full_name: "oxp_36139540-6634-4f0c-9c90-f57efa2432be/crypt/debug" } file = sled-storage/src/manager.rs:1431
6062025-01-08T18:20:12.853ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6072025-01-08T18:20:12.853ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
6082025-01-08T18:20:12.853ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/93dafc63-6f00-46e6-a292-a8413d960ce4/crypt/zone"), full_name: "oxp_93dafc63-6f00-46e6-a292-a8413d960ce4/crypt/zone" } file = sled-storage/src/manager.rs:1431
6092025-01-08T18:20:12.883ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6102025-01-08T18:20:12.883ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6112025-01-08T18:20:12.883ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/93dafc63-6f00-46e6-a292-a8413d960ce4/crypt/zone/oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646"), full_name: "oxp_93dafc63-6f00-46e6-a292-a8413d960ce4/crypt/zone/oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646" } file = sled-storage/src/manager.rs:1431
6122025-01-08T18:20:12.943ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6132025-01-08T18:20:12.943ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
6142025-01-08T18:20:12.943ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c1f40097-34cd-4b0e-8a1d-a4c6931740f6/crypt/debug"), full_name: "oxp_c1f40097-34cd-4b0e-8a1d-a4c6931740f6/crypt/debug" } file = sled-storage/src/manager.rs:1431
6152025-01-08T18:20:12.972ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6162025-01-08T18:20:12.972ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6172025-01-08T18:20:12.972ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/efc412bd-0718-4381-8d37-e5a204c8c3de/crypt/zone/oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a"), full_name: "oxp_efc412bd-0718-4381-8d37-e5a204c8c3de/crypt/zone/oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a" } file = sled-storage/src/manager.rs:1431
6182025-01-08T18:20:13.033ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6192025-01-08T18:20:13.033ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6202025-01-08T18:20:13.033ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/zone/oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81"), full_name: "oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/zone/oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81" } file = sled-storage/src/manager.rs:1431
6212025-01-08T18:20:13.093ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6222025-01-08T18:20:13.093ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6232025-01-08T18:20:13.093ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_bf665b71-2255-48d6-b1fd-c863b4c8e1af/crypt/internal_dns" } file = sled-storage/src/manager.rs:1431
6242025-01-08T18:20:13.142ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6252025-01-08T18:20:13.142ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6262025-01-08T18:20:13.143ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_c1f40097-34cd-4b0e-8a1d-a4c6931740f6/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1431
6272025-01-08T18:20:13.193ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6282025-01-08T18:20:13.193ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6292025-01-08T18:20:13.193ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_4f399814-e4ff-4480-aa92-c4c936f90fd0/crucible" } file = sled-storage/src/manager.rs:1431
6302025-01-08T18:20:13.241ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6312025-01-08T18:20:13.241ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
6322025-01-08T18:20:13.241ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4f399814-e4ff-4480-aa92-c4c936f90fd0/crypt/debug"), full_name: "oxp_4f399814-e4ff-4480-aa92-c4c936f90fd0/crypt/debug" } file = sled-storage/src/manager.rs:1431
6332025-01-08T18:20:13.269ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6342025-01-08T18:20:13.269ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6352025-01-08T18:20:13.269ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/93dafc63-6f00-46e6-a292-a8413d960ce4/crypt/zone/oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda"), full_name: "oxp_93dafc63-6f00-46e6-a292-a8413d960ce4/crypt/zone/oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda" } file = sled-storage/src/manager.rs:1431
6362025-01-08T18:20:13.331ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6372025-01-08T18:20:13.331ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
6382025-01-08T18:20:13.331ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/bf665b71-2255-48d6-b1fd-c863b4c8e1af/crypt/zone"), full_name: "oxp_bf665b71-2255-48d6-b1fd-c863b4c8e1af/crypt/zone" } file = sled-storage/src/manager.rs:1431
6392025-01-08T18:20:13.361ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6402025-01-08T18:20:13.361ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6412025-01-08T18:20:13.361ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/93dafc63-6f00-46e6-a292-a8413d960ce4/crypt/zone/oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4"), full_name: "oxp_93dafc63-6f00-46e6-a292-a8413d960ce4/crypt/zone/oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4" } file = sled-storage/src/manager.rs:1431
6422025-01-08T18:20:13.383ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 10.735863762s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
6432025-01-08T18:20:13.422ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6442025-01-08T18:20:13.422ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6452025-01-08T18:20:13.422ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_0ac8ff68-15ea-4477-a206-5a7619973e85/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1431
6462025-01-08T18:20:13.473ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6472025-01-08T18:20:13.473ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
6482025-01-08T18:20:13.473ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/debug"), full_name: "oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/debug" } file = sled-storage/src/manager.rs:1431
6492025-01-08T18:20:13.502ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6502025-01-08T18:20:13.502ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6512025-01-08T18:20:13.502ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_0ac8ff68-15ea-4477-a206-5a7619973e85/crypt/internal_dns" } file = sled-storage/src/manager.rs:1431
6522025-01-08T18:20:13.553ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6532025-01-08T18:20:13.553ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6542025-01-08T18:20:13.553ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/91e62592-6bd8-45a2-a311-bfef0a5977f4/crypt/zone/oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c"), full_name: "oxp_91e62592-6bd8-45a2-a311-bfef0a5977f4/crypt/zone/oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c" } file = sled-storage/src/manager.rs:1431
6552025-01-08T18:20:13.612ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6562025-01-08T18:20:13.612ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6572025-01-08T18:20:13.612ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/4f399814-e4ff-4480-aa92-c4c936f90fd0/crypt/zone/oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7"), full_name: "oxp_4f399814-e4ff-4480-aa92-c4c936f90fd0/crypt/zone/oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7" } file = sled-storage/src/manager.rs:1431
6582025-01-08T18:20:13.636ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0) file = sled-agent/src/probe_manager.rs:231 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
6592025-01-08T18:20:13.674ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6602025-01-08T18:20:13.674ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
6612025-01-08T18:20:13.674ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/c1f40097-34cd-4b0e-8a1d-a4c6931740f6/crypt/zone"), full_name: "oxp_c1f40097-34cd-4b0e-8a1d-a4c6931740f6/crypt/zone" } file = sled-storage/src/manager.rs:1431
6622025-01-08T18:20:13.703ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/.omicron-datasets.json.tmp file = common/src/ledger.rs:199 request = datasets_ensure
6632025-01-08T18:20:13.703ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/.omicron-datasets.json.tmp file = common/src/ledger.rs:199 request = datasets_ensure
6642025-01-08T18:20:13.704ZINFOSledAgent: datasets ensure: Updated storage file = sled-agent/src/sled_agent.rs:861 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
6652025-01-08T18:20:13.704ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3213928 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:40092 req_id = f2bcb9a7-955b-4ad7-9ac6-8a71a86872de response_code = 200 uri = /datasets
6662025-01-08T18:20:13.736ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: 392d8525-4c31-48fa-8943-c48b0201800f (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 654ed3a1-a574-4551-8bd0-2ab525bdb4a4 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: fa9d7cee-8b9d-425a-afbc-64b19bf54aaa (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
6672025-01-08T18:20:13.736ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:41107
6682025-01-08T18:20:13.736ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
6692025-01-08T18:20:13.736ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }
6702025-01-08T18:20:13.746ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
6712025-01-08T18:20:13.746ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6722025-01-08T18:20:13.755ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
6732025-01-08T18:20:13.755ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }
6742025-01-08T18:20:13.763ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
6752025-01-08T18:20:13.763ZINFOSledAgent (ServiceManager): No ledger in /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-zones.json file = common/src/ledger.rs:188
6762025-01-08T18:20:13.763ZINFOSledAgent (ServiceManager): No ledger in /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-zones.json file = common/src/ledger.rs:188
6772025-01-08T18:20:13.763ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
6782025-01-08T18:20:13.805ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
6792025-01-08T18:20:13.805ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
6802025-01-08T18:20:13.805ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
6812025-01-08T18:20:13.858ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4 file = illumos-utils/src/zone.rs:325
6822025-01-08T18:20:13.862ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f file = illumos-utils/src/zone.rs:325
6832025-01-08T18:20:13.866ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa file = illumos-utils/src/zone.rs:325
6842025-01-08T18:20:13.878ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4 file = illumos-utils/src/zone.rs:363
6852025-01-08T18:20:13.885ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f file = illumos-utils/src/zone.rs:363
6862025-01-08T18:20:13.890ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa file = illumos-utils/src/zone.rs:363
6872025-01-08T18:20:14.638ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0) file = sled-agent/src/probe_manager.rs:231 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
6882025-01-08T18:20:15.445ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService2'/> <propval type="astring" name="gateway" value='fd00:1122:3344:1::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:1::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:1::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:1::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6892025-01-08T18:20:15.924ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:1::, width: 64 }
6902025-01-08T18:20:15.924ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService3'/> <propval type="astring" name="gateway" value='fd00:1122:3344:3::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:3::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:3::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:3::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6912025-01-08T18:20:15.924ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:3::, width: 64 }
6922025-01-08T18:20:15.953ZINFOSledAgent (ServiceManager): refreshed mgd service with new configuration file = sled-agent/src/services.rs:4559
6932025-01-08T18:20:15.953ZINFOSledAgent (ServiceManager): configuring mg-ddm service file = sled-agent/src/services.rs:4565
6942025-01-08T18:20:16.047ZINFOSledAgent (ServiceManager): refreshed mg-ddm service with new configuration file = sled-agent/src/services.rs:4606
6952025-01-08T18:20:16.047ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:370
6962025-01-08T18:20:16.380ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 2349201026162231174 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
6972025-01-08T18:20:16.380ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 6225703372757367109 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
6982025-01-08T18:20:16.383ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 51.084604ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id) file = sled-agent/src/bootstrap/early_networking.rs:389
6992025-01-08T18:20:16.410ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService1'/> <propval type="astring" name="gateway" value='fd00:1122:3344:2::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:2::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:2::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:2::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7002025-01-08T18:20:16.410ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f
7012025-01-08T18:20:16.415ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa
7022025-01-08T18:20:16.419ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:2::, width: 64 }
7032025-01-08T18:20:16.419ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4
7042025-01-08T18:20:16.435ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 108.252738ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id) file = sled-agent/src/bootstrap/early_networking.rs:389
7052025-01-08T18:20:16.550ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:420
7062025-01-08T18:20:16.553ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f") failed: Property not found. retry in 44.349627ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f
7072025-01-08T18:20:16.561ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa") failed: Property not found. retry in 61.937133ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa
7082025-01-08T18:20:16.569ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4") failed: Property not found. retry in 42.167804ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4
7092025-01-08T18:20:16.588ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:717
7102025-01-08T18:20:16.588ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:762
7112025-01-08T18:20:16.589ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:765 uptime = 0
7122025-01-08T18:20:16.589ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.1.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: Some(\n None,\n ),\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n tx_eq: None,\n },\n },\n },\n} file = sled-agent/src/bootstrap/early_networking.rs:444
7132025-01-08T18:20:16.661ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4") failed: Property not found. retry in 140.526371ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4
7142025-01-08T18:20:16.783ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f") failed: Property not found. retry in 88.178084ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f
7152025-01-08T18:20:16.827ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa") failed: Property not found. retry in 69.746162ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa
7162025-01-08T18:20:17.005ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
7172025-01-08T18:20:17.043ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4") failed: Property not found. retry in 203.704365ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4
7182025-01-08T18:20:17.111ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f") failed: Property not found. retry in 244.203307ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f
7192025-01-08T18:20:17.204ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa") failed: Property not found. retry in 263.044682ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa
7202025-01-08T18:20:17.358ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4") failed: Property not found. retry in 280.635452ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4
7212025-01-08T18:20:17.396ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 18250165013537883834 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
7222025-01-08T18:20:17.397ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 2727085820797177907 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
7232025-01-08T18:20:17.398ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f") failed: Property not found. retry in 217.401159ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f
7242025-01-08T18:20:17.407ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:607
7252025-01-08T18:20:17.407ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
7262025-01-08T18:20:17.407ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
7272025-01-08T18:20:17.407ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
7282025-01-08T18:20:17.407ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
7292025-01-08T18:20:17.407ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
7302025-01-08T18:20:17.407ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
7312025-01-08T18:20:17.407ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
7322025-01-08T18:20:17.407ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
7332025-01-08T18:20:17.407ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
7342025-01-08T18:20:17.407ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
7352025-01-08T18:20:17.408ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
7362025-01-08T18:20:17.408ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
7372025-01-08T18:20:17.408ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
7382025-01-08T18:20:17.408ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
7392025-01-08T18:20:17.408ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
7402025-01-08T18:20:17.408ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
7412025-01-08T18:20:17.408ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
7422025-01-08T18:20:17.408ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
7432025-01-08T18:20:17.408ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
7442025-01-08T18:20:17.408ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
7452025-01-08T18:20:17.408ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
7462025-01-08T18:20:17.408ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
7472025-01-08T18:20:17.480ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa") failed: Property not found. retry in 251.493831ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa
7482025-01-08T18:20:17.625ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f") failed: Property not found. retry in 606.931061ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f
7492025-01-08T18:20:17.649ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4") failed: Property not found. retry in 1.098608511s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4
7502025-01-08T18:20:17.741ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa") failed: Property not found. retry in 1.140813862s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa
7512025-01-08T18:20:18.244ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f") failed: Property not found. retry in 838.45899ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f
7522025-01-08T18:20:18.760ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4") failed: Property not found. retry in 1.434493845s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4
7532025-01-08T18:20:18.895ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa") failed: Property not found. retry in 704.574189ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa
7542025-01-08T18:20:19.094ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f") failed: Property not found. retry in 1.349826757s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f
7552025-01-08T18:20:19.610ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa") failed: Property not found. retry in 1.347784567s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa
7562025-01-08T18:20:20.203ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4") failed: Property not found. retry in 1.425851798s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4
7572025-01-08T18:20:20.452ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f") failed: Property not found. retry in 1.405287889s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f
7582025-01-08T18:20:20.966ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa") failed: Property not found. retry in 842.036075ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa
7592025-01-08T18:20:21.641ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4") failed: Property not found. retry in 1.420156782s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4
7602025-01-08T18:20:21.818ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa") failed: Property not found. retry in 1.17552071s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa
7612025-01-08T18:20:21.867ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f") failed: Property not found. retry in 703.909825ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f
7622025-01-08T18:20:23.079ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_internal_dns_654ed3a1-a574-4551-8bd0-2ab525bdb4a4
7632025-01-08T18:20:23.079ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_internal_dns_392d8525-4c31-48fa-8943-c48b0201800f
7642025-01-08T18:20:23.079ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_internal_dns_fa9d7cee-8b9d-425a-afbc-64b19bf54aaa
7652025-01-08T18:20:23.079ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7662025-01-08T18:20:23.079ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7672025-01-08T18:20:23.079ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9343507 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:41107 req_id = af2f5161-c1e5-4605-a958-66eeca9f546c response_code = 204 uri = /omicron-zones
7682025-01-08T18:20:23.081ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:658
7692025-01-08T18:20:23.155ZWARNSledAgent (RSS): failed to write DNS configuration (will retry in 73.770907ms) dns_config_addr = [fd00:1122:3344:1::1]:5353 error_message = Communication Error: error sending request for url (http://[fd00:1122:3344:1::1]:5353/config) file = sled-agent/src/rack_setup/service.rs:678
7702025-01-08T18:20:23.255ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:658
7712025-01-08T18:20:23.309ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:658
7722025-01-08T18:20:23.363ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:694
7732025-01-08T18:20:23.363ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
7742025-01-08T18:20:23.395ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
7752025-01-08T18:20:23.396ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
7762025-01-08T18:20:23.396ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
7772025-01-08T18:20:23.425ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: 392d8525-4c31-48fa-8943-c48b0201800f (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 654ed3a1-a574-4551-8bd0-2ab525bdb4a4 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: fa9d7cee-8b9d-425a-afbc-64b19bf54aaa (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: ddedf876-8456-4d11-91dc-9b39882f74e4 (service), filesystem_pool: Some(ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 717c5259-349b-43db-bdc3-a0ed97cde3bc, kind: Service { id: ddedf876-8456-4d11-91dc-9b39882f74e4 }, name: Name("ntp-ddedf876-8456-4d11-91dc-9b39882f74e4"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
7782025-01-08T18:20:23.426ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:45936
7792025-01-08T18:20:23.426ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
7802025-01-08T18:20:23.426ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }
7812025-01-08T18:20:23.437ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
7822025-01-08T18:20:23.437ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7832025-01-08T18:20:23.447ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
7842025-01-08T18:20:23.447ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }
7852025-01-08T18:20:23.457ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
7862025-01-08T18:20:23.457ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-zones.json file = common/src/ledger.rs:177
7872025-01-08T18:20:23.457ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-zones.json file = common/src/ledger.rs:177
7882025-01-08T18:20:23.457ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
7892025-01-08T18:20:23.468ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
7902025-01-08T18:20:23.500ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
7912025-01-08T18:20:23.500ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
7922025-01-08T18:20:23.500ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
7932025-01-08T18:20:23.530ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte0", ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 204, 188]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 } }) }
7942025-01-08T18:20:23.530ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1275 zone_type = boundary_ntp
7952025-01-08T18:20:23.530ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.169 file = clients/dpd-client/src/lib.rs:83
7962025-01-08T18:20:23.530ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.169 file = clients/dpd-client/src/lib.rs:90
7972025-01-08T18:20:23.530ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.169 file = clients/dpd-client/src/lib.rs:118
7982025-01-08T18:20:23.531ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
7992025-01-08T18:20:23.557ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4 file = illumos-utils/src/zone.rs:325
8002025-01-08T18:20:23.574ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4 file = illumos-utils/src/zone.rs:363
8012025-01-08T18:20:24.204ZINFOSledAgent (ServiceManager): Profile for oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService4'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/chrony-setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="allow" value='fd00:1122:3344:100::/56'/> <propval type="boolean" name="boundary" value='true'/> <propval type="astring" name="boundary_pool" value='boundary-ntp.control-plane.oxide.internal'/> <propval type="astring" name="server" value='0.pool.ntp.org'/> </property_group> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="1.1.1.1" /> <value_node value="9.9.9.9" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/ntp"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte0'/> <propval type="astring" name="gateway" value='172.30.3.1'/> <propval type="astring" name="ip" value='172.30.3.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8022025-01-08T18:20:24.204ZINFOSledAgent (ServiceManager): Booting oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4
8032025-01-08T18:20:24.342ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4") failed: Property not found. retry in 43.759168ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4
8042025-01-08T18:20:24.394ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4") failed: Property not found. retry in 81.768876ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4
8052025-01-08T18:20:24.487ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4") failed: Property not found. retry in 122.188202ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4
8062025-01-08T18:20:24.620ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4") failed: Property not found. retry in 550.455238ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4
8072025-01-08T18:20:25.183ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4") failed: Property not found. retry in 1.088479355s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4
8082025-01-08T18:20:26.280ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4") failed: Property not found. retry in 1.027252555s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4
8092025-01-08T18:20:26.382ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 2349201026162231174 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
8102025-01-08T18:20:26.382ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 6225703372757367109 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
8112025-01-08T18:20:27.318ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4") failed: Property not found. retry in 631.598003ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4
8122025-01-08T18:20:27.398ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 18250165013537883834 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
8132025-01-08T18:20:27.398ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 2727085820797177907 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
8142025-01-08T18:20:27.959ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4") failed: Property not found. retry in 670.690835ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4
8152025-01-08T18:20:28.638ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4") failed: Property not found. retry in 1.084768571s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4
8162025-01-08T18:20:29.744ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4
8172025-01-08T18:20:29.744ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
8182025-01-08T18:20:29.744ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
8192025-01-08T18:20:29.744ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6318791 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:45936 req_id = 0d8cf539-f552-4302-90ee-5f03e07bbdf9 response_code = 204 uri = /omicron-zones
8202025-01-08T18:20:29.747ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:735
8212025-01-08T18:20:29.776ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8222025-01-08T18:20:29.777ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55910
8232025-01-08T18:20:29.786ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3926
8242025-01-08T18:20:29.786ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 9166 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:55910 req_id = 64c5492a-59c3-4bb6-b657-2985aefdfea6 response_code = 500 uri = /timesync
8252025-01-08T18:20:29.789ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8262025-01-08T18:20:29.880ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8272025-01-08T18:20:29.880ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55737
8282025-01-08T18:20:32.597ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 5704809099601365475 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
8292025-01-08T18:20:33.016ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 8520123241479523945 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
8302025-01-08T18:20:33.080ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 5845640364456094245 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
8312025-01-08T18:20:33.889ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3926
8322025-01-08T18:20:33.889ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 4008645 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:55737 req_id = 58172da3-aede-4653-be11-fe6e67c326f2 response_code = 500 uri = /timesync
8332025-01-08T18:20:33.891ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8342025-01-08T18:20:34.026ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8352025-01-08T18:20:34.026ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:65281
8362025-01-08T18:20:34.035ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3926
8372025-01-08T18:20:34.035ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 8903 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:65281 req_id = 957c10a9-3081-4872-80c5-d517a3a5a926 response_code = 500 uri = /timesync
8382025-01-08T18:20:34.038ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8392025-01-08T18:20:34.247ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8402025-01-08T18:20:34.247ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:58803
8412025-01-08T18:20:34.255ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_ddedf876-8456-4d11-91dc-9b39882f74e4': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3926
8422025-01-08T18:20:34.255ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 8197 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:58803 req_id = e88fce63-5c5a-45e0-b308-33565599bab2 response_code = 500 uri = /timesync
8432025-01-08T18:20:34.258ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8442025-01-08T18:20:34.409ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8452025-01-08T18:20:34.409ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:37043
8462025-01-08T18:20:36.418ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2008728 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:37043 req_id = 795c9863-358f-4a1b-92e8-eaf6fd3c7456 response_code = 200 uri = /timesync
8472025-01-08T18:20:36.419ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 2349201026162231174 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
8482025-01-08T18:20:36.419ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 6225703372757367109 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
8492025-01-08T18:20:36.422ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:743
8502025-01-08T18:20:36.422ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8512025-01-08T18:20:36.700ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8522025-01-08T18:20:36.701ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64751
8532025-01-08T18:20:36.710ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9018 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:64751 req_id = 24c4f4b5-d128-4800-8e0b-9161b1e9c2fd response_code = 200 uri = /timesync
8542025-01-08T18:20:36.712ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:743
8552025-01-08T18:20:36.712ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8562025-01-08T18:20:37.143ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8572025-01-08T18:20:37.143ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:61155
8582025-01-08T18:20:37.152ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8519 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:61155 req_id = e33ba3e5-0c15-459e-a291-6460a38f119e response_code = 200 uri = /timesync
8592025-01-08T18:20:37.154ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:743
8602025-01-08T18:20:37.154ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8612025-01-08T18:20:37.133ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 18250165013537883834 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
8622025-01-08T18:20:37.133ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 2727085820797177907 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
8632025-01-08T18:20:37.276ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8642025-01-08T18:20:37.277ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:61967
8652025-01-08T18:20:37.285ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8140 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:61967 req_id = ecb5a32f-9e92-4b26-b4be-627a3f347e83 response_code = 200 uri = /timesync
8662025-01-08T18:20:37.288ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:743
8672025-01-08T18:20:37.288ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8682025-01-08T18:20:37.769ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8692025-01-08T18:20:37.769ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:56018
8702025-01-08T18:20:37.777ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7869 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:56018 req_id = 2266f015-aaf6-43fd-a064-8109c50cd26f response_code = 200 uri = /timesync
8712025-01-08T18:20:37.779ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:743
8722025-01-08T18:20:37.779ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8732025-01-08T18:20:38.193ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8742025-01-08T18:20:38.193ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:58137
8752025-01-08T18:20:38.201ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7502 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:58137 req_id = 81324351-9fd3-41a2-ac18-ea5b9bb25ce1 response_code = 200 uri = /timesync
8762025-01-08T18:20:38.205ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1736360436.8937972, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:743
8772025-01-08T18:20:38.205ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8782025-01-08T18:20:38.711ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8792025-01-08T18:20:38.711ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60095
8802025-01-08T18:20:38.719ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7818 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:60095 req_id = 432744fa-7ee6-4c77-bed1-11c6e829791f response_code = 200 uri = /timesync
8812025-01-08T18:20:38.721ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1736360436.8937972, correction: 0.037668686 } file = sled-agent/src/rack_setup/service.rs:743
8822025-01-08T18:20:38.721ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8832025-01-08T18:20:38.997ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 139.009065ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
8842025-01-08T18:20:39.309ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8852025-01-08T18:20:39.309ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:40863
8862025-01-08T18:20:39.318ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8966 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:40863 req_id = 5d2b882e-6bff-48d3-9aa8-e80be84447b1 response_code = 200 uri = /timesync
8872025-01-08T18:20:39.321ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1736360436.8937972, correction: 0.038127568 } file = sled-agent/src/rack_setup/service.rs:743
8882025-01-08T18:20:39.321ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8892025-01-08T18:20:39.588ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 2023923433290177370 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
8902025-01-08T18:20:39.589ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 16539421261258384925 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
8912025-01-08T18:20:40.457ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8922025-01-08T18:20:40.458ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62556
8932025-01-08T18:20:40.479ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 21031 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:62556 req_id = 43912b10-ea42-478c-beb4-a0318a65e503 response_code = 200 uri = /timesync
8942025-01-08T18:20:40.484ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 2616771612, ip_addr: 155.248.196.28, stratum: 3, ref_time: 1736360440.244084, correction: 0.034998924 } file = sled-agent/src/rack_setup/service.rs:743
8952025-01-08T18:20:40.484ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1392
8962025-01-08T18:20:40.515ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 392d8525-4c31-48fa-8943-c48b0201800f (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 654ed3a1-a574-4551-8bd0-2ab525bdb4a4 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: fa9d7cee-8b9d-425a-afbc-64b19bf54aaa (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: ddedf876-8456-4d11-91dc-9b39882f74e4 (service), filesystem_pool: Some(ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 717c5259-349b-43db-bdc3-a0ed97cde3bc, kind: Service { id: ddedf876-8456-4d11-91dc-9b39882f74e4 }, name: Name("ntp-ddedf876-8456-4d11-91dc-9b39882f74e4"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: a9ba89b7-0e4a-4156-b7af-623bdf9eed77 (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } } } }, OmicronZoneConfig { id: 7db60888-d89b-431c-9600-46f13cdc6d58 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } } } }, OmicronZoneConfig { id: d9316a13-f819-469a-bdc2-7716f0dd92e1 (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } } } }, OmicronZoneConfig { id: 0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 (service), filesystem_pool: Some(ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External } } } }, OmicronZoneConfig { id: f14f6e05-f479-462f-9c78-458b849e5795 (service), filesystem_pool: Some(ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
8972025-01-08T18:20:40.515ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:38727
8982025-01-08T18:20:40.515ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
8992025-01-08T18:20:40.515ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }
9002025-01-08T18:20:40.526ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9012025-01-08T18:20:40.526ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9022025-01-08T18:20:40.536ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9032025-01-08T18:20:40.536ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }
9042025-01-08T18:20:40.545ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9052025-01-08T18:20:40.545ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }
9062025-01-08T18:20:40.553ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9072025-01-08T18:20:40.553ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9082025-01-08T18:20:40.563ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9092025-01-08T18:20:40.563ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }
9102025-01-08T18:20:40.571ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9112025-01-08T18:20:40.571ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9122025-01-08T18:20:40.580ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9132025-01-08T18:20:40.580ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9142025-01-08T18:20:40.589ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9152025-01-08T18:20:40.589ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-zones.json file = common/src/ledger.rs:177
9162025-01-08T18:20:40.589ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-zones.json file = common/src/ledger.rs:177
9172025-01-08T18:20:40.589ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9182025-01-08T18:20:40.620ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
9192025-01-08T18:20:40.620ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
9202025-01-08T18:20:40.620ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
9212025-01-08T18:20:40.620ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
9222025-01-08T18:20:40.620ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
9232025-01-08T18:20:40.620ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795", TimeNotSynchronized), ("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58", TimeNotSynchronized), ("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7", TimeNotSynchronized), ("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77", TimeNotSynchronized), ("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 104670 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:38727 req_id = e262cac2-52d4-42da-8acc-f909b944ebb7 response_code = 500 uri = /omicron-zones
9242025-01-08T18:20:40.620ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "e262cac2-52d4-42da-8acc-f909b944ebb7", "content-length": "124", "date": "Wed, 08 Jan 2025 18:20:40 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "e262cac2-52d4-42da-8acc-f909b944ebb7" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 62.078862ms sled_address = [fd00:1122:3344:101::1]:12345
9252025-01-08T18:20:40.683ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 392d8525-4c31-48fa-8943-c48b0201800f (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 654ed3a1-a574-4551-8bd0-2ab525bdb4a4 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: fa9d7cee-8b9d-425a-afbc-64b19bf54aaa (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: ddedf876-8456-4d11-91dc-9b39882f74e4 (service), filesystem_pool: Some(ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 717c5259-349b-43db-bdc3-a0ed97cde3bc, kind: Service { id: ddedf876-8456-4d11-91dc-9b39882f74e4 }, name: Name("ntp-ddedf876-8456-4d11-91dc-9b39882f74e4"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: a9ba89b7-0e4a-4156-b7af-623bdf9eed77 (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } } } }, OmicronZoneConfig { id: 7db60888-d89b-431c-9600-46f13cdc6d58 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } } } }, OmicronZoneConfig { id: d9316a13-f819-469a-bdc2-7716f0dd92e1 (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } } } }, OmicronZoneConfig { id: 0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 (service), filesystem_pool: Some(ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External } } } }, OmicronZoneConfig { id: f14f6e05-f479-462f-9c78-458b849e5795 (service), filesystem_pool: Some(ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
9262025-01-08T18:20:40.683ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9272025-01-08T18:20:40.684ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }
9282025-01-08T18:20:40.693ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9292025-01-08T18:20:40.693ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9302025-01-08T18:20:40.702ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9312025-01-08T18:20:40.702ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }
9322025-01-08T18:20:40.711ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9332025-01-08T18:20:40.711ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }
9342025-01-08T18:20:40.720ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9352025-01-08T18:20:40.721ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9362025-01-08T18:20:40.729ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9372025-01-08T18:20:40.729ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }
9382025-01-08T18:20:40.739ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9392025-01-08T18:20:40.739ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9402025-01-08T18:20:40.748ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9412025-01-08T18:20:40.748ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9422025-01-08T18:20:40.757ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9432025-01-08T18:20:40.757ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-zones.json file = common/src/ledger.rs:177
9442025-01-08T18:20:40.757ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-zones.json file = common/src/ledger.rs:177
9452025-01-08T18:20:40.757ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9462025-01-08T18:20:40.788ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
9472025-01-08T18:20:40.788ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
9482025-01-08T18:20:40.788ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
9492025-01-08T18:20:40.788ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
9502025-01-08T18:20:40.788ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
9512025-01-08T18:20:40.788ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58", TimeNotSynchronized), ("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1", TimeNotSynchronized), ("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77", TimeNotSynchronized), ("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7", TimeNotSynchronized), ("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 104619 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:38727 req_id = 12b99b06-9d46-4c63-ae31-8db5928ad8a0 response_code = 500 uri = /omicron-zones
9522025-01-08T18:20:40.788ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "12b99b06-9d46-4c63-ae31-8db5928ad8a0", "content-length": "124", "date": "Wed, 08 Jan 2025 18:20:40 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "12b99b06-9d46-4c63-ae31-8db5928ad8a0" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 104.414932ms sled_address = [fd00:1122:3344:101::1]:12345
9532025-01-08T18:20:40.894ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 392d8525-4c31-48fa-8943-c48b0201800f (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 654ed3a1-a574-4551-8bd0-2ab525bdb4a4 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: fa9d7cee-8b9d-425a-afbc-64b19bf54aaa (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: ddedf876-8456-4d11-91dc-9b39882f74e4 (service), filesystem_pool: Some(ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 717c5259-349b-43db-bdc3-a0ed97cde3bc, kind: Service { id: ddedf876-8456-4d11-91dc-9b39882f74e4 }, name: Name("ntp-ddedf876-8456-4d11-91dc-9b39882f74e4"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: a9ba89b7-0e4a-4156-b7af-623bdf9eed77 (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } } } }, OmicronZoneConfig { id: 7db60888-d89b-431c-9600-46f13cdc6d58 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } } } }, OmicronZoneConfig { id: d9316a13-f819-469a-bdc2-7716f0dd92e1 (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } } } }, OmicronZoneConfig { id: 0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 (service), filesystem_pool: Some(ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External } } } }, OmicronZoneConfig { id: f14f6e05-f479-462f-9c78-458b849e5795 (service), filesystem_pool: Some(ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
9542025-01-08T18:20:40.894ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9552025-01-08T18:20:40.894ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }
9562025-01-08T18:20:40.904ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9572025-01-08T18:20:40.904ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9582025-01-08T18:20:40.913ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9592025-01-08T18:20:40.913ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }
9602025-01-08T18:20:40.922ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9612025-01-08T18:20:40.922ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }
9622025-01-08T18:20:40.931ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9632025-01-08T18:20:40.931ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9642025-01-08T18:20:40.941ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9652025-01-08T18:20:40.941ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }
9662025-01-08T18:20:40.950ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9672025-01-08T18:20:40.950ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9682025-01-08T18:20:40.959ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9692025-01-08T18:20:40.959ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9702025-01-08T18:20:40.968ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9712025-01-08T18:20:40.968ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-zones.json file = common/src/ledger.rs:177
9722025-01-08T18:20:40.968ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-zones.json file = common/src/ledger.rs:177
9732025-01-08T18:20:40.968ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9742025-01-08T18:20:41.002ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
9752025-01-08T18:20:41.002ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
9762025-01-08T18:20:41.002ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
9772025-01-08T18:20:41.002ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
9782025-01-08T18:20:41.002ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
9792025-01-08T18:20:41.002ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795", TimeNotSynchronized), ("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58", TimeNotSynchronized), ("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7", TimeNotSynchronized), ("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1", TimeNotSynchronized), ("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 107639 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:38727 req_id = 84024464-5d28-4483-80ae-6f7b10e5278c response_code = 500 uri = /omicron-zones
9802025-01-08T18:20:41.002ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "84024464-5d28-4483-80ae-6f7b10e5278c", "content-length": "124", "date": "Wed, 08 Jan 2025 18:20:40 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "84024464-5d28-4483-80ae-6f7b10e5278c" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 80.967676ms sled_address = [fd00:1122:3344:101::1]:12345
9812025-01-08T18:20:41.085ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 392d8525-4c31-48fa-8943-c48b0201800f (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 654ed3a1-a574-4551-8bd0-2ab525bdb4a4 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: fa9d7cee-8b9d-425a-afbc-64b19bf54aaa (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: ddedf876-8456-4d11-91dc-9b39882f74e4 (service), filesystem_pool: Some(ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 717c5259-349b-43db-bdc3-a0ed97cde3bc, kind: Service { id: ddedf876-8456-4d11-91dc-9b39882f74e4 }, name: Name("ntp-ddedf876-8456-4d11-91dc-9b39882f74e4"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: a9ba89b7-0e4a-4156-b7af-623bdf9eed77 (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } } } }, OmicronZoneConfig { id: 7db60888-d89b-431c-9600-46f13cdc6d58 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } } } }, OmicronZoneConfig { id: d9316a13-f819-469a-bdc2-7716f0dd92e1 (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } } } }, OmicronZoneConfig { id: 0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 (service), filesystem_pool: Some(ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External } } } }, OmicronZoneConfig { id: f14f6e05-f479-462f-9c78-458b849e5795 (service), filesystem_pool: Some(ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
9822025-01-08T18:20:41.085ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9832025-01-08T18:20:41.085ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }
9842025-01-08T18:20:41.094ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9852025-01-08T18:20:41.094ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9862025-01-08T18:20:41.103ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9872025-01-08T18:20:41.103ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }
9882025-01-08T18:20:41.111ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9892025-01-08T18:20:41.111ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }
9902025-01-08T18:20:41.120ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9912025-01-08T18:20:41.120ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9922025-01-08T18:20:41.129ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9932025-01-08T18:20:41.129ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }
9942025-01-08T18:20:41.138ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9952025-01-08T18:20:41.138ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9962025-01-08T18:20:41.147ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9972025-01-08T18:20:41.147ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9982025-01-08T18:20:41.156ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9992025-01-08T18:20:41.156ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-zones.json file = common/src/ledger.rs:177
10002025-01-08T18:20:41.156ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-zones.json file = common/src/ledger.rs:177
10012025-01-08T18:20:41.157ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10022025-01-08T18:20:41.189ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
10032025-01-08T18:20:41.189ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
10042025-01-08T18:20:41.189ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
10052025-01-08T18:20:41.189ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
10062025-01-08T18:20:41.189ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
10072025-01-08T18:20:41.189ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58", TimeNotSynchronized), ("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1", TimeNotSynchronized), ("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7", TimeNotSynchronized), ("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795", TimeNotSynchronized), ("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 104103 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:38727 req_id = 3ef6334f-9c03-4197-a6a5-80b894ef337c response_code = 500 uri = /omicron-zones
10082025-01-08T18:20:41.189ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "3ef6334f-9c03-4197-a6a5-80b894ef337c", "content-length": "124", "date": "Wed, 08 Jan 2025 18:20:40 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "3ef6334f-9c03-4197-a6a5-80b894ef337c" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 218.746292ms sled_address = [fd00:1122:3344:101::1]:12345
10092025-01-08T18:20:41.409ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 392d8525-4c31-48fa-8943-c48b0201800f (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 654ed3a1-a574-4551-8bd0-2ab525bdb4a4 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: fa9d7cee-8b9d-425a-afbc-64b19bf54aaa (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: ddedf876-8456-4d11-91dc-9b39882f74e4 (service), filesystem_pool: Some(ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 717c5259-349b-43db-bdc3-a0ed97cde3bc, kind: Service { id: ddedf876-8456-4d11-91dc-9b39882f74e4 }, name: Name("ntp-ddedf876-8456-4d11-91dc-9b39882f74e4"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: a9ba89b7-0e4a-4156-b7af-623bdf9eed77 (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } } } }, OmicronZoneConfig { id: 7db60888-d89b-431c-9600-46f13cdc6d58 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } } } }, OmicronZoneConfig { id: d9316a13-f819-469a-bdc2-7716f0dd92e1 (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } } } }, OmicronZoneConfig { id: 0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 (service), filesystem_pool: Some(ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External } } } }, OmicronZoneConfig { id: f14f6e05-f479-462f-9c78-458b849e5795 (service), filesystem_pool: Some(ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
10102025-01-08T18:20:41.409ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10112025-01-08T18:20:41.409ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }
10122025-01-08T18:20:41.419ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10132025-01-08T18:20:41.419ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10142025-01-08T18:20:41.427ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10152025-01-08T18:20:41.427ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }
10162025-01-08T18:20:41.437ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10172025-01-08T18:20:41.437ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }
10182025-01-08T18:20:41.446ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10192025-01-08T18:20:41.446ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10202025-01-08T18:20:41.455ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10212025-01-08T18:20:41.455ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }
10222025-01-08T18:20:41.464ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10232025-01-08T18:20:41.464ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10242025-01-08T18:20:41.474ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10252025-01-08T18:20:41.474ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10262025-01-08T18:20:41.483ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10272025-01-08T18:20:41.483ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-zones.json file = common/src/ledger.rs:177
10282025-01-08T18:20:41.483ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-zones.json file = common/src/ledger.rs:177
10292025-01-08T18:20:41.483ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10302025-01-08T18:20:41.515ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
10312025-01-08T18:20:41.515ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
10322025-01-08T18:20:41.515ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
10332025-01-08T18:20:41.515ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
10342025-01-08T18:20:41.515ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
10352025-01-08T18:20:41.515ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1", TimeNotSynchronized), ("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58", TimeNotSynchronized), ("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77", TimeNotSynchronized), ("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795", TimeNotSynchronized), ("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 105611 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:38727 req_id = b9a15fca-875e-4ee2-80c7-31052ef1e97b response_code = 500 uri = /omicron-zones
10362025-01-08T18:20:41.515ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "b9a15fca-875e-4ee2-80c7-31052ef1e97b", "content-length": "124", "date": "Wed, 08 Jan 2025 18:20:41 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "b9a15fca-875e-4ee2-80c7-31052ef1e97b" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 234.350859ms sled_address = [fd00:1122:3344:101::1]:12345
10372025-01-08T18:20:41.751ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 392d8525-4c31-48fa-8943-c48b0201800f (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 654ed3a1-a574-4551-8bd0-2ab525bdb4a4 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: fa9d7cee-8b9d-425a-afbc-64b19bf54aaa (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: ddedf876-8456-4d11-91dc-9b39882f74e4 (service), filesystem_pool: Some(ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 717c5259-349b-43db-bdc3-a0ed97cde3bc, kind: Service { id: ddedf876-8456-4d11-91dc-9b39882f74e4 }, name: Name("ntp-ddedf876-8456-4d11-91dc-9b39882f74e4"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: a9ba89b7-0e4a-4156-b7af-623bdf9eed77 (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } } } }, OmicronZoneConfig { id: 7db60888-d89b-431c-9600-46f13cdc6d58 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } } } }, OmicronZoneConfig { id: d9316a13-f819-469a-bdc2-7716f0dd92e1 (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } } } }, OmicronZoneConfig { id: 0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 (service), filesystem_pool: Some(ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External } } } }, OmicronZoneConfig { id: f14f6e05-f479-462f-9c78-458b849e5795 (service), filesystem_pool: Some(ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
10382025-01-08T18:20:41.751ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10392025-01-08T18:20:41.751ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }
10402025-01-08T18:20:41.761ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10412025-01-08T18:20:41.761ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10422025-01-08T18:20:41.770ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10432025-01-08T18:20:41.770ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }
10442025-01-08T18:20:41.778ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10452025-01-08T18:20:41.778ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }
10462025-01-08T18:20:41.787ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10472025-01-08T18:20:41.787ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10482025-01-08T18:20:41.797ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10492025-01-08T18:20:41.797ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }
10502025-01-08T18:20:41.805ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10512025-01-08T18:20:41.805ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10522025-01-08T18:20:41.814ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10532025-01-08T18:20:41.814ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10542025-01-08T18:20:41.823ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10552025-01-08T18:20:41.824ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-zones.json file = common/src/ledger.rs:177
10562025-01-08T18:20:41.824ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-zones.json file = common/src/ledger.rs:177
10572025-01-08T18:20:41.824ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10582025-01-08T18:20:41.856ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
10592025-01-08T18:20:41.856ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
10602025-01-08T18:20:41.856ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
10612025-01-08T18:20:41.856ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
10622025-01-08T18:20:41.856ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
10632025-01-08T18:20:41.856ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77", TimeNotSynchronized), ("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7", TimeNotSynchronized), ("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795", TimeNotSynchronized), ("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58", TimeNotSynchronized), ("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 104841 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:38727 req_id = 2387aa3c-8ae2-41f9-aaf9-1a05ebdb0cd2 response_code = 500 uri = /omicron-zones
10642025-01-08T18:20:41.857ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "2387aa3c-8ae2-41f9-aaf9-1a05ebdb0cd2", "content-length": "124", "date": "Wed, 08 Jan 2025 18:20:41 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "2387aa3c-8ae2-41f9-aaf9-1a05ebdb0cd2" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 316.463829ms sled_address = [fd00:1122:3344:101::1]:12345
10652025-01-08T18:20:42.175ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 392d8525-4c31-48fa-8943-c48b0201800f (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 654ed3a1-a574-4551-8bd0-2ab525bdb4a4 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: fa9d7cee-8b9d-425a-afbc-64b19bf54aaa (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: ddedf876-8456-4d11-91dc-9b39882f74e4 (service), filesystem_pool: Some(ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 717c5259-349b-43db-bdc3-a0ed97cde3bc, kind: Service { id: ddedf876-8456-4d11-91dc-9b39882f74e4 }, name: Name("ntp-ddedf876-8456-4d11-91dc-9b39882f74e4"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: a9ba89b7-0e4a-4156-b7af-623bdf9eed77 (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } } } }, OmicronZoneConfig { id: 7db60888-d89b-431c-9600-46f13cdc6d58 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } } } }, OmicronZoneConfig { id: d9316a13-f819-469a-bdc2-7716f0dd92e1 (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } } } }, OmicronZoneConfig { id: 0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 (service), filesystem_pool: Some(ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External } } } }, OmicronZoneConfig { id: f14f6e05-f479-462f-9c78-458b849e5795 (service), filesystem_pool: Some(ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
10662025-01-08T18:20:42.175ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10672025-01-08T18:20:42.175ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }
10682025-01-08T18:20:42.185ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10692025-01-08T18:20:42.185ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10702025-01-08T18:20:42.193ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10712025-01-08T18:20:42.193ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }
10722025-01-08T18:20:42.202ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10732025-01-08T18:20:42.202ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }
10742025-01-08T18:20:42.211ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10752025-01-08T18:20:42.211ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10762025-01-08T18:20:42.221ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10772025-01-08T18:20:42.221ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }
10782025-01-08T18:20:42.230ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10792025-01-08T18:20:42.230ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10802025-01-08T18:20:42.238ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10812025-01-08T18:20:42.239ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10822025-01-08T18:20:42.247ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10832025-01-08T18:20:42.247ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-zones.json file = common/src/ledger.rs:177
10842025-01-08T18:20:42.247ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-zones.json file = common/src/ledger.rs:177
10852025-01-08T18:20:42.248ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10862025-01-08T18:20:42.280ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
10872025-01-08T18:20:42.281ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
10882025-01-08T18:20:42.281ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
10892025-01-08T18:20:42.281ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
10902025-01-08T18:20:42.281ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
10912025-01-08T18:20:42.281ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77", TimeNotSynchronized), ("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58", TimeNotSynchronized), ("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7", TimeNotSynchronized), ("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795", TimeNotSynchronized), ("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 105362 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:38727 req_id = 4d3ef914-87ac-4e98-a8a9-4a16455be528 response_code = 500 uri = /omicron-zones
10922025-01-08T18:20:42.281ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "4d3ef914-87ac-4e98-a8a9-4a16455be528", "content-length": "124", "date": "Wed, 08 Jan 2025 18:20:41 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "4d3ef914-87ac-4e98-a8a9-4a16455be528" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 441.550013ms sled_address = [fd00:1122:3344:101::1]:12345
10932025-01-08T18:20:42.725ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 392d8525-4c31-48fa-8943-c48b0201800f (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 654ed3a1-a574-4551-8bd0-2ab525bdb4a4 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: fa9d7cee-8b9d-425a-afbc-64b19bf54aaa (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: ddedf876-8456-4d11-91dc-9b39882f74e4 (service), filesystem_pool: Some(ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 717c5259-349b-43db-bdc3-a0ed97cde3bc, kind: Service { id: ddedf876-8456-4d11-91dc-9b39882f74e4 }, name: Name("ntp-ddedf876-8456-4d11-91dc-9b39882f74e4"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: a9ba89b7-0e4a-4156-b7af-623bdf9eed77 (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } } } }, OmicronZoneConfig { id: 7db60888-d89b-431c-9600-46f13cdc6d58 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } } } }, OmicronZoneConfig { id: d9316a13-f819-469a-bdc2-7716f0dd92e1 (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } } } }, OmicronZoneConfig { id: 0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 (service), filesystem_pool: Some(ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External } } } }, OmicronZoneConfig { id: f14f6e05-f479-462f-9c78-458b849e5795 (service), filesystem_pool: Some(ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
10942025-01-08T18:20:42.725ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10952025-01-08T18:20:42.725ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }
10962025-01-08T18:20:42.734ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10972025-01-08T18:20:42.734ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10982025-01-08T18:20:42.743ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10992025-01-08T18:20:42.743ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }
11002025-01-08T18:20:42.753ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
11012025-01-08T18:20:42.753ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }
11022025-01-08T18:20:42.761ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
11032025-01-08T18:20:42.761ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }
11042025-01-08T18:20:42.770ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
11052025-01-08T18:20:42.770ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }
11062025-01-08T18:20:42.779ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
11072025-01-08T18:20:42.779ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
11082025-01-08T18:20:42.788ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
11092025-01-08T18:20:42.788ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
11102025-01-08T18:20:42.797ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
11112025-01-08T18:20:42.797ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-zones.json file = common/src/ledger.rs:177
11122025-01-08T18:20:42.798ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-zones.json file = common/src/ledger.rs:177
11132025-01-08T18:20:42.798ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
11142025-01-08T18:20:42.886ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
11152025-01-08T18:20:42.886ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
11162025-01-08T18:20:42.886ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
11172025-01-08T18:20:42.886ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
11182025-01-08T18:20:42.886ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
11192025-01-08T18:20:42.981ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795 file = illumos-utils/src/zone.rs:325
11202025-01-08T18:20:42.985ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 file = illumos-utils/src/zone.rs:325
11212025-01-08T18:20:42.989ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77 file = illumos-utils/src/zone.rs:325
11222025-01-08T18:20:42.993ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58 file = illumos-utils/src/zone.rs:325
11232025-01-08T18:20:42.998ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1 file = illumos-utils/src/zone.rs:325
11242025-01-08T18:20:43.003ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795 file = illumos-utils/src/zone.rs:363
11252025-01-08T18:20:43.008ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 file = illumos-utils/src/zone.rs:363
11262025-01-08T18:20:43.013ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77 file = illumos-utils/src/zone.rs:363
11272025-01-08T18:20:43.017ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58 file = illumos-utils/src/zone.rs:363
11282025-01-08T18:20:43.022ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1 file = illumos-utils/src/zone.rs:363
11292025-01-08T18:20:46.081ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService7'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::3'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='a9ba89b7-0e4a-4156-b7af-623bdf9eed77'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::3]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11302025-01-08T18:20:46.081ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
11312025-01-08T18:20:46.143ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService9'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='d9316a13-f819-469a-bdc2-7716f0dd92e1'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::5]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11322025-01-08T18:20:46.143ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
11332025-01-08T18:20:46.150ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService8'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::4'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='7db60888-d89b-431c-9600-46f13cdc6d58'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::4]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11342025-01-08T18:20:46.150ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
11352025-01-08T18:20:46.168ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService5'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='f14f6e05-f479-462f-9c78-458b849e5795'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::7]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11362025-01-08T18:20:46.168ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
11372025-01-08T18:20:46.179ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService6'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='0aa15cc3-4e5b-4ebf-b647-89a40d11cae7'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::6]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11382025-01-08T18:20:46.179ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
11392025-01-08T18:20:46.234ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77") failed: Property not found. retry in 67.617053ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
11402025-01-08T18:20:46.285ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58") failed: Property not found. retry in 31.896963ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
11412025-01-08T18:20:46.294ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1") failed: Property not found. retry in 37.526206ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
11422025-01-08T18:20:46.319ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77") failed: Property not found. retry in 88.368427ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
11432025-01-08T18:20:46.331ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795") failed: Property not found. retry in 63.691077ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
11442025-01-08T18:20:46.343ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58") failed: Property not found. retry in 146.356941ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
11452025-01-08T18:20:46.353ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7") failed: Property not found. retry in 29.60569ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
11462025-01-08T18:20:46.392ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1") failed: Property not found. retry in 52.850579ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
11472025-01-08T18:20:46.541ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7") failed: Property not found. retry in 149.445656ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
11482025-01-08T18:20:46.542ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:34915
11492025-01-08T18:20:46.598ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 156 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:34915 req_id = 251e857a-372c-4e0e-a132-bd7bea161531 response_code = 200 uri = /baseboard
11502025-01-08T18:20:46.610ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795") failed: Property not found. retry in 147.08741ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
11512025-01-08T18:20:46.880ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77") failed: Property not found. retry in 215.409556ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
11522025-01-08T18:20:47.224ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58") failed: Property not found. retry in 216.29466ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
11532025-01-08T18:20:47.384ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1") failed: Property not found. retry in 236.100693ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
11542025-01-08T18:20:47.439ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795") failed: Property not found. retry in 298.969272ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
11552025-01-08T18:20:47.465ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7") failed: Property not found. retry in 222.999719ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
11562025-01-08T18:20:47.488ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77") failed: Property not found. retry in 296.605471ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
11572025-01-08T18:20:47.510ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58") failed: Property not found. retry in 291.299024ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
11582025-01-08T18:20:47.638ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1") failed: Property not found. retry in 317.527132ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
11592025-01-08T18:20:47.704ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7") failed: Property not found. retry in 544.352615ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
11602025-01-08T18:20:47.756ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795") failed: Property not found. retry in 250.741598ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
11612025-01-08T18:20:47.798ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77") failed: Property not found. retry in 968.744053ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
11622025-01-08T18:20:47.815ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58") failed: Property not found. retry in 868.75039ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
11632025-01-08T18:20:47.975ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1") failed: Property not found. retry in 1.01117424s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
11642025-01-08T18:20:48.025ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795") failed: Property not found. retry in 891.832008ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
11652025-01-08T18:20:48.265ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7") failed: Property not found. retry in 932.168253ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
11662025-01-08T18:20:48.698ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58") failed: Property not found. retry in 1.399185047s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
11672025-01-08T18:20:48.782ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77") failed: Property not found. retry in 557.551382ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
11682025-01-08T18:20:48.932ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795") failed: Property not found. retry in 1.247576738s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
11692025-01-08T18:20:49.000ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1") failed: Property not found. retry in 1.350022797s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
11702025-01-08T18:20:49.212ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7") failed: Property not found. retry in 975.769963ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
11712025-01-08T18:20:49.352ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77") failed: Property not found. retry in 656.511783ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
11722025-01-08T18:20:50.020ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77") failed: Property not found. retry in 1.25998006s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
11732025-01-08T18:20:50.110ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58") failed: Property not found. retry in 608.57703ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
11742025-01-08T18:20:50.192ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795") failed: Property not found. retry in 1.060871158s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
11752025-01-08T18:20:50.201ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7") failed: Property not found. retry in 583.246923ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
11762025-01-08T18:20:50.364ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1") failed: Property not found. retry in 837.068352ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
11772025-01-08T18:20:50.729ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58") failed: Property not found. retry in 1.065206747s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
11782025-01-08T18:20:50.795ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7") failed: Property not found. retry in 1.264048845s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
11792025-01-08T18:20:51.212ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1") failed: Property not found. retry in 730.255345ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
11802025-01-08T18:20:51.265ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795") failed: Property not found. retry in 1.008297965s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
11812025-01-08T18:20:51.292ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77") failed: Property not found. retry in 565.297559ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
11822025-01-08T18:20:51.807ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58") failed: Property not found. retry in 551.159909ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
11832025-01-08T18:20:51.868ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77") failed: Property not found. retry in 711.989315ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
11842025-01-08T18:20:51.953ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1") failed: Property not found. retry in 1.088537998s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
11852025-01-08T18:20:52.071ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7") failed: Property not found. retry in 1.289327684s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
11862025-01-08T18:20:52.285ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795") failed: Property not found. retry in 1.339566951s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
11872025-01-08T18:20:52.369ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58") failed: Property not found. retry in 789.324601ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
11882025-01-08T18:20:52.593ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77") failed: Property not found. retry in 974.744702ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
11892025-01-08T18:20:53.054ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1") failed: Property not found. retry in 1.462922878s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
11902025-01-08T18:20:53.171ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58") failed: Property not found. retry in 872.806976ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
11912025-01-08T18:20:53.373ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7") failed: Property not found. retry in 722.562618ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
11922025-01-08T18:20:53.579ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77") failed: Property not found. retry in 1.276334452s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
11932025-01-08T18:20:53.638ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795") failed: Property not found. retry in 630.662444ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
11942025-01-08T18:20:54.054ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58") failed: Property not found. retry in 887.644576ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
11952025-01-08T18:20:54.106ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7") failed: Property not found. retry in 530.472399ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
11962025-01-08T18:20:54.212ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 747.719106ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
11972025-01-08T18:20:54.277ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795") failed: Property not found. retry in 1.041895125s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
11982025-01-08T18:20:54.526ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1") failed: Property not found. retry in 528.714704ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
11992025-01-08T18:20:54.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7") failed: Property not found. retry in 1.234836823s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
12002025-01-08T18:20:54.866ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77") failed: Property not found. retry in 591.877255ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
12012025-01-08T18:20:54.952ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58") failed: Property not found. retry in 1.255837185s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
12022025-01-08T18:20:55.065ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1") failed: Property not found. retry in 582.463097ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
12032025-01-08T18:20:55.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795") failed: Property not found. retry in 1.489049695s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
12042025-01-08T18:20:55.468ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77") failed: Property not found. retry in 889.109725ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
12052025-01-08T18:20:55.657ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1") failed: Property not found. retry in 687.558424ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
12062025-01-08T18:20:55.889ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7") failed: Property not found. retry in 939.750983ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
12072025-01-08T18:20:56.218ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58") failed: Property not found. retry in 1.097527018s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
12082025-01-08T18:20:56.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1") failed: Property not found. retry in 584.009977ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
12092025-01-08T18:20:56.365ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77") failed: Property not found. retry in 1.036532806s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
12102025-01-08T18:20:56.827ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795") failed: Property not found. retry in 522.581193ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
12112025-01-08T18:20:56.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7") failed: Property not found. retry in 868.688619ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
12122025-01-08T18:20:56.948ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1") failed: Property not found. retry in 1.45996957s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
12132025-01-08T18:20:57.325ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58") failed: Property not found. retry in 1.490830975s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
12142025-01-08T18:20:57.357ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795") failed: Property not found. retry in 1.371937941s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
12152025-01-08T18:20:57.413ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77") failed: Property not found. retry in 1.337001699s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
12162025-01-08T18:20:57.715ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7") failed: Property not found. retry in 1.421781746s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
12172025-01-08T18:20:59.157ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_cockroachdb_f14f6e05-f479-462f-9c78-458b849e5795
12182025-01-08T18:20:59.157ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7
12192025-01-08T18:20:59.157ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_cockroachdb_a9ba89b7-0e4a-4156-b7af-623bdf9eed77
12202025-01-08T18:20:59.157ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_cockroachdb_7db60888-d89b-431c-9600-46f13cdc6d58
12212025-01-08T18:20:59.157ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_cockroachdb_d9316a13-f819-469a-bdc2-7716f0dd92e1
12222025-01-08T18:20:59.157ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
12232025-01-08T18:20:59.157ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
12242025-01-08T18:20:59.157ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 16405120 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:38727 req_id = fd04cea0-e0eb-48b1-a198-042289367655 response_code = 204 uri = /omicron-zones
12252025-01-08T18:20:59.191ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64643
12262025-01-08T18:20:59.218ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 26827 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64643 req_id = a4677c1c-d8b9-4fe1-8e00-729889985b27 response_code = 500 uri = /cockroachdb
12272025-01-08T18:20:59.218ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "a4677c1c-d8b9-4fe1-8e00-729889985b27", "content-length": "124", "date": "Wed, 08 Jan 2025 18:20:59 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "a4677c1c-d8b9-4fe1-8e00-729889985b27" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 129.486464ms
12282025-01-08T18:20:59.374ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 25604 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64643 req_id = c87954db-0d12-4ddc-9dd3-1ca8fc4f5c7e response_code = 500 uri = /cockroachdb
12292025-01-08T18:20:59.374ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "c87954db-0d12-4ddc-9dd3-1ca8fc4f5c7e", "content-length": "124", "date": "Wed, 08 Jan 2025 18:20:59 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "c87954db-0d12-4ddc-9dd3-1ca8fc4f5c7e" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 84.328523ms
12302025-01-08T18:20:59.486ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 27084 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64643 req_id = 106868ee-dbf7-4c4d-8aae-4683461e6bf9 response_code = 500 uri = /cockroachdb
12312025-01-08T18:20:59.487ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "106868ee-dbf7-4c4d-8aae-4683461e6bf9", "content-length": "124", "date": "Wed, 08 Jan 2025 18:20:59 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "106868ee-dbf7-4c4d-8aae-4683461e6bf9" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 147.420462ms
12322025-01-08T18:20:59.662ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 26235 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64643 req_id = d2991986-e4cc-467d-880c-76834a49bb2b response_code = 500 uri = /cockroachdb
12332025-01-08T18:20:59.662ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "d2991986-e4cc-467d-880c-76834a49bb2b", "content-length": "124", "date": "Wed, 08 Jan 2025 18:20:59 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "d2991986-e4cc-467d-880c-76834a49bb2b" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 108.01991ms
12342025-01-08T18:20:59.796ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 24902 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64643 req_id = ed096f1e-48d6-46a0-b1c4-bf9efb20a9d9 response_code = 500 uri = /cockroachdb
12352025-01-08T18:20:59.796ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "ed096f1e-48d6-46a0-b1c4-bf9efb20a9d9", "content-length": "124", "date": "Wed, 08 Jan 2025 18:20:59 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "ed096f1e-48d6-46a0-b1c4-bf9efb20a9d9" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 104.507678ms
12362025-01-08T18:20:59.927ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 25585 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64643 req_id = c4137297-0573-4fda-b854-4b43e02b10d5 response_code = 500 uri = /cockroachdb
12372025-01-08T18:20:59.927ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "c4137297-0573-4fda-b854-4b43e02b10d5", "content-length": "124", "date": "Wed, 08 Jan 2025 18:20:59 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "c4137297-0573-4fda-b854-4b43e02b10d5" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 168.211455ms
12382025-01-08T18:21:00.123ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 26721 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64643 req_id = 43b1c6c4-779c-4a98-9145-158079677467 response_code = 500 uri = /cockroachdb
12392025-01-08T18:21:00.123ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "43b1c6c4-779c-4a98-9145-158079677467", "content-length": "124", "date": "Wed, 08 Jan 2025 18:20:59 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "43b1c6c4-779c-4a98-9145-158079677467" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 275.198193ms
12402025-01-08T18:21:00.426ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 26033 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64643 req_id = a9f07635-4be9-474d-b71d-f08195810e2a response_code = 500 uri = /cockroachdb
12412025-01-08T18:21:00.426ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "a9f07635-4be9-474d-b71d-f08195810e2a", "content-length": "124", "date": "Wed, 08 Jan 2025 18:21:00 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "a9f07635-4be9-474d-b71d-f08195810e2a" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 498.646128ms
12422025-01-08T18:21:00.954ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 28358 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64643 req_id = 1073aeb2-ac1c-4cb9-8c25-47192272c4e3 response_code = 500 uri = /cockroachdb
12432025-01-08T18:21:00.954ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "1073aeb2-ac1c-4cb9-8c25-47192272c4e3", "content-length": "124", "date": "Wed, 08 Jan 2025 18:21:00 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "1073aeb2-ac1c-4cb9-8c25-47192272c4e3" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 503.587412ms
12442025-01-08T18:21:01.486ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 27134 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64643 req_id = c25d160e-4b76-44b0-9b50-6ffd76fb89d5 response_code = 500 uri = /cockroachdb
12452025-01-08T18:21:01.486ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "c25d160e-4b76-44b0-9b50-6ffd76fb89d5", "content-length": "124", "date": "Wed, 08 Jan 2025 18:21:01 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "c25d160e-4b76-44b0-9b50-6ffd76fb89d5" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 271.887109ms
12462025-01-08T18:21:01.788ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 29315 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64643 req_id = bde41b62-9ca4-4b8d-89e9-d8bce869a304 response_code = 500 uri = /cockroachdb
12472025-01-08T18:21:01.788ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "bde41b62-9ca4-4b8d-89e9-d8bce869a304", "content-length": "124", "date": "Wed, 08 Jan 2025 18:21:01 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "bde41b62-9ca4-4b8d-89e9-d8bce869a304" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 314.770288ms
12482025-01-08T18:21:02.135ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::6]:32221 file = sled-agent/src/services.rs:3769
12492025-01-08T18:21:29.879ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3787
12502025-01-08T18:21:35.918ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:3810
12512025-01-08T18:21:35.919ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 33813943 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64643 req_id = ad19b9a5-59fa-4b8d-bb84-1073ca3f0f26 response_code = 204 uri = /cockroachdb
12522025-01-08T18:21:35.922ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:65498
12532025-01-08T18:21:35.928ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 807.742353ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
12542025-01-08T18:21:35.963ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: 392d8525-4c31-48fa-8943-c48b0201800f (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 654ed3a1-a574-4551-8bd0-2ab525bdb4a4 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: fa9d7cee-8b9d-425a-afbc-64b19bf54aaa (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: ddedf876-8456-4d11-91dc-9b39882f74e4 (service), filesystem_pool: Some(ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 717c5259-349b-43db-bdc3-a0ed97cde3bc, kind: Service { id: ddedf876-8456-4d11-91dc-9b39882f74e4 }, name: Name("ntp-ddedf876-8456-4d11-91dc-9b39882f74e4"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 204, 188])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.1.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: a9ba89b7-0e4a-4156-b7af-623bdf9eed77 (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } } } }, OmicronZoneConfig { id: 7db60888-d89b-431c-9600-46f13cdc6d58 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } } } }, OmicronZoneConfig { id: d9316a13-f819-469a-bdc2-7716f0dd92e1 (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } } } }, OmicronZoneConfig { id: 0aa15cc3-4e5b-4ebf-b647-89a40d11cae7 (service), filesystem_pool: Some(ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External } } } }, OmicronZoneConfig { id: f14f6e05-f479-462f-9c78-458b849e5795 (service), filesystem_pool: Some(ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 3963284d-f267-448b-bfbb-a55402c7ef81 (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.1.164:53, nic: NetworkInterface { id: ce1a1751-7375-4744-b6f0-690b02866c95, kind: Service { id: 3963284d-f267-448b-bfbb-a55402c7ef81 }, name: Name("external-dns-3963284d-f267-448b-bfbb-a55402c7ef81"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 116])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: 1c3a7513-ed19-483c-872f-3377ba80f37f (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.1.165:53, nic: NetworkInterface { id: 0041456a-da2f-4ef8-bdf7-8e18ca9d8279, kind: Service { id: 1c3a7513-ed19-483c-872f-3377ba80f37f }, name: Name("external-dns-1c3a7513-ed19-483c-872f-3377ba80f37f"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 154, 170])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: cef4a321-387e-44a1-92c9-23673073cdd8 (service), filesystem_pool: Some(ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.1.166, nic: NetworkInterface { id: 2beb6270-a9c4-41a0-b3ab-eeff6ddd3582, kind: Service { id: cef4a321-387e-44a1-92c9-23673073cdd8 }, name: Name("nexus-cef4a321-387e-44a1-92c9-23673073cdd8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 113])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 099ecd74-9dbb-4a7f-94ce-79177e3bb646 (service), filesystem_pool: Some(ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.1.167, nic: NetworkInterface { id: c656ac3d-ffdc-434e-863c-8efcfbd82ab4, kind: Service { id: 099ecd74-9dbb-4a7f-94ce-79177e3bb646 }, name: Name("nexus-099ecd74-9dbb-4a7f-94ce-79177e3bb646"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 171])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: a28f8263-3fdd-4cd1-a79d-6ef15080a80e (service), filesystem_pool: Some(ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.1.168, nic: NetworkInterface { id: bfb8fa95-f4f1-4ec3-bb77-0e8adbc6ac68, kind: Service { id: a28f8263-3fdd-4cd1-a79d-6ef15080a80e }, name: Name("nexus-a28f8263-3fdd-4cd1-a79d-6ef15080a80e"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 4daad4eb-a44a-4284-aed7-ecda6bd3827a (service), filesystem_pool: Some(ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }), zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: e958ccc4-cfdf-45a9-9788-badac857001f (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } } } }, OmicronZoneConfig { id: 5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd (service), filesystem_pool: Some(ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: 95ba58cc-ac8e-4184-ab2a-6133ffeee501 (service), filesystem_pool: Some(ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: 3df2e090-e138-45b7-ae0c-befa35147766 (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: 258c2c4c-1016-4d80-b3d4-737729d8a8bb (service), filesystem_pool: Some(ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External } } } }, OmicronZoneConfig { id: 4b41895a-9eb9-4013-a351-9044e18f7fc3 (service), filesystem_pool: Some(ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External } } } }, OmicronZoneConfig { id: 72b64c60-6adc-43c6-9811-00f42b56c8f3 (service), filesystem_pool: Some(ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External } } } }, OmicronZoneConfig { id: 288f7daa-e3bd-4928-a1ba-e69d732272fd (service), filesystem_pool: Some(ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 0dcfd88e-be05-40ed-9457-11e0fb9fdb63 (service), filesystem_pool: Some(ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 77527a2c-7c89-4a87-b0cc-afb1ad48e17a (service), filesystem_pool: Some(ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External } } } }, OmicronZoneConfig { id: 45d1606c-760e-49e1-93f5-68d2f8a0056c (service), filesystem_pool: Some(ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External } } } }, OmicronZoneConfig { id: 33bf58e9-44fc-44da-9328-86fd24a66bda (service), filesystem_pool: Some(ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1e710058-8813-4cc1-b005-f322f467107b (service), filesystem_pool: Some(ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
12552025-01-08T18:21:35.964ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:53881
12562025-01-08T18:21:35.965ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
12572025-01-08T18:21:35.965ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: InternalDns }, responder: ... }
12582025-01-08T18:21:35.981ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
12592025-01-08T18:21:35.981ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: InternalDns }, responder: ... }
12602025-01-08T18:21:35.991ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
12612025-01-08T18:21:35.991ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: InternalDns }, responder: ... }
12622025-01-08T18:21:36.002ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
12632025-01-08T18:21:36.002ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Cockroach }, responder: ... }
12642025-01-08T18:21:36.012ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
12652025-01-08T18:21:36.012ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Cockroach }, responder: ... }
12662025-01-08T18:21:36.022ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
12672025-01-08T18:21:36.022ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Cockroach }, responder: ... }
12682025-01-08T18:21:36.032ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
12692025-01-08T18:21:36.032ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
12702025-01-08T18:21:36.042ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
12712025-01-08T18:21:36.042ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
12722025-01-08T18:21:36.051ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
12732025-01-08T18:21:36.051ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: ExternalDns }, responder: ... }
12742025-01-08T18:21:36.062ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
12752025-01-08T18:21:36.062ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
12762025-01-08T18:21:36.071ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Clickhouse }, responder: ... }) file = sled-storage/src/manager.rs:607
12772025-01-08T18:21:36.071ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Clickhouse }, responder: ... }
12782025-01-08T18:21:36.081ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12792025-01-08T18:21:36.081ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, kind: Crucible }, responder: ... }
12802025-01-08T18:21:36.091ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12812025-01-08T18:21:36.091ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, kind: Crucible }, responder: ... }
12822025-01-08T18:21:36.102ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12832025-01-08T18:21:36.102ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, kind: Crucible }, responder: ... }
12842025-01-08T18:21:36.112ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12852025-01-08T18:21:36.112ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, kind: Crucible }, responder: ... }
12862025-01-08T18:21:36.122ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12872025-01-08T18:21:36.122ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, kind: Crucible }, responder: ... }
12882025-01-08T18:21:36.131ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12892025-01-08T18:21:36.131ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }, kind: Crucible }, responder: ... }
12902025-01-08T18:21:36.142ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12912025-01-08T18:21:36.142ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }, kind: Crucible }, responder: ... }
12922025-01-08T18:21:36.153ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12932025-01-08T18:21:36.153ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, kind: Crucible }, responder: ... }
12942025-01-08T18:21:36.163ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12952025-01-08T18:21:36.163ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External }, kind: Crucible }, responder: ... }
12962025-01-08T18:21:36.173ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12972025-01-08T18:21:36.173ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-zones.json file = common/src/ledger.rs:177
12982025-01-08T18:21:36.173ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-zones.json file = common/src/ledger.rs:177
12992025-01-08T18:21:36.173ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13002025-01-08T18:21:36.321ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
13012025-01-08T18:21:36.322ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13022025-01-08T18:21:36.322ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13032025-01-08T18:21:36.322ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13042025-01-08T18:21:36.322ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13052025-01-08T18:21:36.370ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
13062025-01-08T18:21:36.370ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13072025-01-08T18:21:36.370ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13082025-01-08T18:21:36.370ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13092025-01-08T18:21:36.372ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13102025-01-08T18:21:36.570ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
13112025-01-08T18:21:36.570ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13122025-01-08T18:21:36.570ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13132025-01-08T18:21:36.570ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13142025-01-08T18:21:36.570ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13152025-01-08T18:21:36.570ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13162025-01-08T18:21:36.583ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
13172025-01-08T18:21:36.583ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13182025-01-08T18:21:36.585ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
13192025-01-08T18:21:36.585ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3 file = illumos-utils/src/zone.rs:325
13202025-01-08T18:21:36.590ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3 file = illumos-utils/src/zone.rs:325
13212025-01-08T18:21:36.595ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c file = illumos-utils/src/zone.rs:325
13222025-01-08T18:21:36.600ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a file = illumos-utils/src/zone.rs:325
13232025-01-08T18:21:36.606ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb file = illumos-utils/src/zone.rs:325
13242025-01-08T18:21:36.611ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda file = illumos-utils/src/zone.rs:325
13252025-01-08T18:21:36.616ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b file = illumos-utils/src/zone.rs:325
13262025-01-08T18:21:36.621ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501 file = illumos-utils/src/zone.rs:325
13272025-01-08T18:21:36.743ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3 file = illumos-utils/src/zone.rs:363
13282025-01-08T18:21:36.748ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3 file = illumos-utils/src/zone.rs:363
13292025-01-08T18:21:36.753ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c file = illumos-utils/src/zone.rs:363
13302025-01-08T18:21:36.758ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a file = illumos-utils/src/zone.rs:363
13312025-01-08T18:21:36.763ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb file = illumos-utils/src/zone.rs:363
13322025-01-08T18:21:36.768ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda file = illumos-utils/src/zone.rs:363
13332025-01-08T18:21:36.773ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b file = illumos-utils/src/zone.rs:363
13342025-01-08T18:21:36.779ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501 file = illumos-utils/src/zone.rs:363
13352025-01-08T18:21:36.785ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd file = illumos-utils/src/zone.rs:325
13362025-01-08T18:21:36.791ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63 file = illumos-utils/src/zone.rs:325
13372025-01-08T18:21:36.798ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd file = illumos-utils/src/zone.rs:325
13382025-01-08T18:21:36.805ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766 file = illumos-utils/src/zone.rs:325
13392025-01-08T18:21:36.812ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a file = illumos-utils/src/zone.rs:325
13402025-01-08T18:21:36.824ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f file = illumos-utils/src/zone.rs:325
13412025-01-08T18:21:36.834ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd file = illumos-utils/src/zone.rs:363
13422025-01-08T18:21:36.842ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63 file = illumos-utils/src/zone.rs:363
13432025-01-08T18:21:36.850ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd file = illumos-utils/src/zone.rs:363
13442025-01-08T18:21:36.860ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766 file = illumos-utils/src/zone.rs:363
13452025-01-08T18:21:36.869ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a file = illumos-utils/src/zone.rs:363
13462025-01-08T18:21:36.880ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f file = illumos-utils/src/zone.rs:363
13472025-01-08T18:21:36.962ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
13482025-01-08T18:21:37.024ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
13492025-01-08T18:21:37.087ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
13502025-01-08T18:21:37.145ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
13512025-01-08T18:21:37.200ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
13522025-01-08T18:21:37.202ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
13532025-01-08T18:21:37.202ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
13542025-01-08T18:21:37.258ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte1", ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 139, 116]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
13552025-01-08T18:21:37.258ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1275 zone_type = external_dns
13562025-01-08T18:21:37.259ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
13572025-01-08T18:21:37.259ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
13582025-01-08T18:21:37.317ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte2", ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 154, 170]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
13592025-01-08T18:21:37.317ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1275 zone_type = external_dns
13602025-01-08T18:21:37.321ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
13612025-01-08T18:21:37.321ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
13622025-01-08T18:21:37.452ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte3", ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 182, 113]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
13632025-01-08T18:21:37.452ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1275 zone_type = nexus
13642025-01-08T18:21:37.453ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.164 file = clients/dpd-client/src/lib.rs:83
13652025-01-08T18:21:37.455ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.164 file = clients/dpd-client/src/lib.rs:90
13662025-01-08T18:21:37.455ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.165 file = clients/dpd-client/src/lib.rs:83
13672025-01-08T18:21:37.455ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.165 file = clients/dpd-client/src/lib.rs:90
13682025-01-08T18:21:37.457ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
13692025-01-08T18:21:37.457ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
13702025-01-08T18:21:37.530ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte4", ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 168, 171]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
13712025-01-08T18:21:37.530ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1275 zone_type = nexus
13722025-01-08T18:21:37.531ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
13732025-01-08T18:21:37.531ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
13742025-01-08T18:21:37.587ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte5", ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 239, 87]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
13752025-01-08T18:21:37.587ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1275 zone_type = nexus
13762025-01-08T18:21:37.588ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.164 file = clients/dpd-client/src/lib.rs:118
13772025-01-08T18:21:37.589ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.165 file = clients/dpd-client/src/lib.rs:118
13782025-01-08T18:21:37.591ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13792025-01-08T18:21:37.591ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13802025-01-08T18:21:37.682ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.166 file = clients/dpd-client/src/lib.rs:83
13812025-01-08T18:21:37.682ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.166 file = clients/dpd-client/src/lib.rs:90
13822025-01-08T18:21:37.682ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81 file = illumos-utils/src/zone.rs:325
13832025-01-08T18:21:37.692ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.167 file = clients/dpd-client/src/lib.rs:83
13842025-01-08T18:21:37.692ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.167 file = clients/dpd-client/src/lib.rs:90
13852025-01-08T18:21:37.692ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.1.168 file = clients/dpd-client/src/lib.rs:83
13862025-01-08T18:21:37.692ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.1.168 file = clients/dpd-client/src/lib.rs:90
13872025-01-08T18:21:37.692ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.166 file = clients/dpd-client/src/lib.rs:118
13882025-01-08T18:21:37.694ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13892025-01-08T18:21:37.695ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.167 file = clients/dpd-client/src/lib.rs:118
13902025-01-08T18:21:37.698ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.1.168 file = clients/dpd-client/src/lib.rs:118
13912025-01-08T18:21:37.701ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13922025-01-08T18:21:37.701ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13932025-01-08T18:21:37.737ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81 file = illumos-utils/src/zone.rs:363
13942025-01-08T18:21:37.750ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f file = illumos-utils/src/zone.rs:325
13952025-01-08T18:21:37.873ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8 file = illumos-utils/src/zone.rs:325
13962025-01-08T18:21:37.881ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f file = illumos-utils/src/zone.rs:363
13972025-01-08T18:21:37.888ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646 file = illumos-utils/src/zone.rs:325
13982025-01-08T18:21:37.896ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e file = illumos-utils/src/zone.rs:325
13992025-01-08T18:21:37.905ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8 file = illumos-utils/src/zone.rs:363
14002025-01-08T18:21:37.965ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646 file = illumos-utils/src/zone.rs:363
14012025-01-08T18:21:37.987ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e file = illumos-utils/src/zone.rs:363
14022025-01-08T18:21:40.804ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService17'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::10'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::10'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14032025-01-08T18:21:40.804ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService20'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::f'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::f'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14042025-01-08T18:21:40.804ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501
14052025-01-08T18:21:40.808ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd
14062025-01-08T18:21:40.819ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService21'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::11'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::11'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14072025-01-08T18:21:40.819ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766
14082025-01-08T18:21:40.865ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService18'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::15'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_4f399814-e4ff-4480-aa92-c4c936f90fd0/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::15'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='5a32ae95-0938-4ee4-b6f7-fde2372602cd'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14092025-01-08T18:21:40.865ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService19'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::16'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_c1f40097-34cd-4b0e-8a1d-a4c6931740f6/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::16'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='ea36634e-fb9a-4ce4-9f87-83ccb85430ab'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14102025-01-08T18:21:40.865ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService22'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::17'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_efc412bd-0718-4381-8d37-e5a204c8c3de/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::17'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='db8845a4-38e1-49ff-af5f-85d560742002'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14112025-01-08T18:21:40.865ZINFOSledAgent (ServiceManager): Booting oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd
14122025-01-08T18:21:40.868ZINFOSledAgent (ServiceManager): Booting oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63
14132025-01-08T18:21:40.896ZINFOSledAgent (ServiceManager): Booting oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a
14142025-01-08T18:21:40.908ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService10'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::13'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_0ac8ff68-15ea-4477-a206-5a7619973e85/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::13'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='6152b797-16d2-4084-82e2-f764e1cf78d8'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14152025-01-08T18:21:40.908ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService11'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::14'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_bf665b71-2255-48d6-b1fd-c863b4c8e1af/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::14'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='e9f9e886-233a-492e-b113-85184070fd3f'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14162025-01-08T18:21:40.908ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService12'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::18'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_91e62592-6bd8-45a2-a311-bfef0a5977f4/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::18'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='10776a9a-549a-48bb-8836-aba763964a94'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14172025-01-08T18:21:40.909ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService15'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::19'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_93dafc63-6f00-46e6-a292-a8413d960ce4/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::19'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='c03a9eb7-a621-4903-ae6e-ffaf712fb78c'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14182025-01-08T18:21:40.909ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService16'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_36139540-6634-4f0c-9c90-f57efa2432be/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1a'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='281f8b29-bbfc-4dcd-b3f4-6061282ce653'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14192025-01-08T18:21:40.910ZINFOSledAgent (ServiceManager): Booting oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3
14202025-01-08T18:21:44.091ZINFOSledAgent (ServiceManager): Booting oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3
14212025-01-08T18:21:44.144ZINFOSledAgent (ServiceManager): Booting oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c
14222025-01-08T18:21:44.164ZINFOSledAgent (ServiceManager): Profile for oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService13'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::d'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/oximeter"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="id" value='4daad4eb-a44a-4284-aed7-ecda6bd3827a'/> <propval type="astring" name="address" value='[fd00:1122:3344:101::d]:12223'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14232025-01-08T18:21:44.166ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService24'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::8'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte1'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::8]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.5:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14242025-01-08T18:21:44.168ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService14'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::12'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::12'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='929096d8-59a1-4844-bd40-7dea5ead1247'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14252025-01-08T18:21:44.169ZINFOSledAgent (ServiceManager): Booting oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda
14262025-01-08T18:21:44.190ZINFOSledAgent (ServiceManager): Booting oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b
14272025-01-08T18:21:44.211ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService25'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::9'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte2'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::9]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.6:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14282025-01-08T18:21:44.261ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501") failed: Property not found. retry in 74.720314ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501
14292025-01-08T18:21:44.294ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd") failed: Property not found. retry in 48.382669ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd
14302025-01-08T18:21:44.316ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63") failed: Property not found. retry in 64.35458ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63
14312025-01-08T18:21:44.336ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd") failed: Property not found. retry in 69.843627ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd
14322025-01-08T18:21:44.387ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766") failed: Property not found. retry in 52.079371ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766
14332025-01-08T18:21:44.410ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a") failed: Property not found. retry in 40.921582ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a
14342025-01-08T18:21:44.411ZINFOSledAgent (ServiceManager): Booting oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a
14352025-01-08T18:21:44.426ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81
14362025-01-08T18:21:44.440ZINFOSledAgent (ServiceManager): Booting oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb
14372025-01-08T18:21:44.454ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f
14382025-01-08T18:21:44.486ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501") failed: Property not found. retry in 121.545874ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501
14392025-01-08T18:21:44.502ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd") failed: Property not found. retry in 116.570949ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd
14402025-01-08T18:21:44.518ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63") failed: Property not found. retry in 147.850229ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63
14412025-01-08T18:21:44.533ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd") failed: Property not found. retry in 144.097139ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd
14422025-01-08T18:21:44.547ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766") failed: Property not found. retry in 106.094213ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766
14432025-01-08T18:21:44.560ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a") failed: Property not found. retry in 91.35788ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a
14442025-01-08T18:21:44.578ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3") failed: Property not found. retry in 40.03683ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3
14452025-01-08T18:21:44.590ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3") failed: Property not found. retry in 72.84036ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3
14462025-01-08T18:21:44.599ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c") failed: Property not found. retry in 70.452909ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c
14472025-01-08T18:21:44.609ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda") failed: Property not found. retry in 68.78075ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda
14482025-01-08T18:21:44.621ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b") failed: Property not found. retry in 27.291164ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b
14492025-01-08T18:21:44.632ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501") failed: Property not found. retry in 221.037622ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501
14502025-01-08T18:21:44.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd") failed: Property not found. retry in 196.592334ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd
14512025-01-08T18:21:44.656ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3") failed: Property not found. retry in 119.273897ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3
14522025-01-08T18:21:44.665ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b") failed: Property not found. retry in 94.752567ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b
14532025-01-08T18:21:44.675ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766") failed: Property not found. retry in 205.329342ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766
14542025-01-08T18:21:44.693ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a") failed: Property not found. retry in 204.627594ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a
14552025-01-08T18:21:44.694ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService26'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte3'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14562025-01-08T18:21:44.713ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3") failed: Property not found. retry in 130.475106ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3
14572025-01-08T18:21:44.733ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c") failed: Property not found. retry in 93.639401ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c
14582025-01-08T18:21:44.750ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a") failed: Property not found. retry in 26.83646ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a
14592025-01-08T18:21:44.762ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda") failed: Property not found. retry in 54.669554ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda
14602025-01-08T18:21:44.771ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b") failed: Property not found. retry in 251.217985ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b
14612025-01-08T18:21:44.781ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f") failed: Property not found. retry in 27.417513ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f
14622025-01-08T18:21:44.791ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63") failed: Property not found. retry in 168.044252ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63
14632025-01-08T18:21:44.800ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd") failed: Property not found. retry in 187.48869ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd
14642025-01-08T18:21:44.800ZINFOSledAgent (ServiceManager): Booting oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
14652025-01-08T18:21:44.819ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3") failed: Property not found. retry in 156.028955ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3
14662025-01-08T18:21:44.828ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a") failed: Property not found. retry in 100.615238ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a
14672025-01-08T18:21:44.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda") failed: Property not found. retry in 248.541721ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda
14682025-01-08T18:21:44.847ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f") failed: Property not found. retry in 86.953445ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f
14692025-01-08T18:21:44.858ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd") failed: Property not found. retry in 334.221782ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd
14702025-01-08T18:21:44.869ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3") failed: Property not found. retry in 281.497995ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3
14712025-01-08T18:21:44.881ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c") failed: Property not found. retry in 221.34602ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c
14722025-01-08T18:21:44.892ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501") failed: Property not found. retry in 389.574399ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501
14732025-01-08T18:21:44.902ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766") failed: Property not found. retry in 336.111528ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766
14742025-01-08T18:21:44.911ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a") failed: Property not found. retry in 391.658809ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a
14752025-01-08T18:21:44.939ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a") failed: Property not found. retry in 263.633382ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a
14762025-01-08T18:21:44.951ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f") failed: Property not found. retry in 224.402169ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f
14772025-01-08T18:21:44.971ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63") failed: Property not found. retry in 322.042611ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63
14782025-01-08T18:21:44.987ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3") failed: Property not found. retry in 328.767579ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3
14792025-01-08T18:21:44.998ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd") failed: Property not found. retry in 460.457771ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd
14802025-01-08T18:21:45.010ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8") failed: Property not found. retry in 71.473109ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
14812025-01-08T18:21:45.036ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b") failed: Property not found. retry in 321.688103ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b
14822025-01-08T18:21:45.093ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8") failed: Property not found. retry in 97.552005ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
14832025-01-08T18:21:45.104ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda") failed: Property not found. retry in 508.471851ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda
14842025-01-08T18:21:45.115ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c") failed: Property not found. retry in 507.640392ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c
14852025-01-08T18:21:45.162ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3") failed: Property not found. retry in 336.574253ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3
14862025-01-08T18:21:45.188ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f") failed: Property not found. retry in 576.147633ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f
14872025-01-08T18:21:45.202ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8") failed: Property not found. retry in 293.617061ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
14882025-01-08T18:21:45.214ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd") failed: Property not found. retry in 611.213031ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd
14892025-01-08T18:21:45.226ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a") failed: Property not found. retry in 504.642503ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a
14902025-01-08T18:21:45.252ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766") failed: Property not found. retry in 666.566316ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766
14912025-01-08T18:21:45.293ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501") failed: Property not found. retry in 733.563191ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501
14922025-01-08T18:21:45.303ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63") failed: Property not found. retry in 1.03022804s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63
14932025-01-08T18:21:45.304ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService27'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte4'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14942025-01-08T18:21:45.304ZINFOSledAgent (ServiceManager): Booting oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646
14952025-01-08T18:21:45.320ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a") failed: Property not found. retry in 463.673957ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a
14962025-01-08T18:21:45.328ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3") failed: Property not found. retry in 485.895028ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3
14972025-01-08T18:21:45.329ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService28'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::c'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte5'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14982025-01-08T18:21:45.329ZINFOSledAgent (ServiceManager): Booting oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e
14992025-01-08T18:21:45.370ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b") failed: Property not found. retry in 1.186833411s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b
15002025-01-08T18:21:45.474ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd") failed: Property not found. retry in 832.007773ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd
15012025-01-08T18:21:45.512ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8") failed: Property not found. retry in 371.482717ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
15022025-01-08T18:21:45.523ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3") failed: Property not found. retry in 967.008488ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3
15032025-01-08T18:21:45.531ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646") failed: Property not found. retry in 34.489334ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646
15042025-01-08T18:21:45.540ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e") failed: Property not found. retry in 32.39765ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e
15052025-01-08T18:21:45.575ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646") failed: Property not found. retry in 121.337126ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646
15062025-01-08T18:21:45.586ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e") failed: Property not found. retry in 67.358525ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e
15072025-01-08T18:21:45.625ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda") failed: Property not found. retry in 633.905524ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda
15082025-01-08T18:21:45.684ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c") failed: Property not found. retry in 634.754249ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c
15092025-01-08T18:21:45.718ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e") failed: Property not found. retry in 255.936482ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e
15102025-01-08T18:21:45.730ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646") failed: Property not found. retry in 182.100213ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646
15112025-01-08T18:21:45.743ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a") failed: Property not found. retry in 1.104463402s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a
15122025-01-08T18:21:45.777ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f") failed: Property not found. retry in 848.1338ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f
15132025-01-08T18:21:45.795ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a") failed: Property not found. retry in 772.866466ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a
15142025-01-08T18:21:45.829ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3") failed: Property not found. retry in 639.339246ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3
15152025-01-08T18:21:45.843ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd") failed: Property not found. retry in 1.150322459s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd
15162025-01-08T18:21:45.896ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8") failed: Property not found. retry in 893.979774ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
15172025-01-08T18:21:45.925ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646") failed: Property not found. retry in 513.606039ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646
15182025-01-08T18:21:45.937ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766") failed: Property not found. retry in 1.256783556s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766
15192025-01-08T18:21:46.025ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e") failed: Property not found. retry in 208.378999ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e
15202025-01-08T18:21:46.047ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501") failed: Property not found. retry in 510.859792ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501
15212025-01-08T18:21:46.256ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e") failed: Property not found. retry in 567.588878ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e
15222025-01-08T18:21:46.276ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81") failed: Property not found. retry in 39.82077ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81
15232025-01-08T18:21:46.298ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda") failed: Property not found. retry in 813.090001ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda
15242025-01-08T18:21:46.324ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd") failed: Property not found. retry in 1.005073946s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd
15252025-01-08T18:21:46.340ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c") failed: Property not found. retry in 696.69441ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c
15262025-01-08T18:21:46.355ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81") failed: Property not found. retry in 86.072808ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81
15272025-01-08T18:21:46.426ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb") failed: Property not found. retry in 25.367918ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb
15282025-01-08T18:21:46.441ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63") failed: Property not found. retry in 667.029668ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63
15292025-01-08T18:21:46.456ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646") failed: Property not found. retry in 546.756438ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646
15302025-01-08T18:21:46.468ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81") failed: Property not found. retry in 103.619042ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81
15312025-01-08T18:21:46.479ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb") failed: Property not found. retry in 99.359017ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb
15322025-01-08T18:21:46.489ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3") failed: Property not found. retry in 519.820909ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3
15332025-01-08T18:21:46.504ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3") failed: Property not found. retry in 1.080104258s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3
15342025-01-08T18:21:46.570ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b") failed: Property not found. retry in 957.72307ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b
15352025-01-08T18:21:46.582ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501") failed: Property not found. retry in 1.030409344s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501
15362025-01-08T18:21:46.593ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a") failed: Property not found. retry in 1.477241215s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a
15372025-01-08T18:21:46.605ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81") failed: Property not found. retry in 317.532102ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81
15382025-01-08T18:21:46.617ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb") failed: Property not found. retry in 206.412523ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb
15392025-01-08T18:21:46.638ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f") failed: Property not found. retry in 1.345614467s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f
15402025-01-08T18:21:46.806ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8") failed: Property not found. retry in 528.780359ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
15412025-01-08T18:21:46.834ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb") failed: Property not found. retry in 455.815192ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb
15422025-01-08T18:21:46.844ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e") failed: Property not found. retry in 1.209074697s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e
15432025-01-08T18:21:46.859ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a") failed: Property not found. retry in 809.982596ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a
15442025-01-08T18:21:46.938ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81") failed: Property not found. retry in 571.970763ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81
15452025-01-08T18:21:47.012ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd") failed: Property not found. retry in 950.4941ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd
15462025-01-08T18:21:47.029ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3") failed: Property not found. retry in 1.26151313s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3
15472025-01-08T18:21:47.045ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646") failed: Property not found. retry in 1.495851283s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646
15482025-01-08T18:21:47.058ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c") failed: Property not found. retry in 1.484682282s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c
15492025-01-08T18:21:47.128ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63") failed: Property not found. retry in 747.453382ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63
15502025-01-08T18:21:47.156ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda") failed: Property not found. retry in 1.245488881s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda
15512025-01-08T18:21:47.212ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766") failed: Property not found. retry in 1.36015841s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766
15522025-01-08T18:21:47.309ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb") failed: Property not found. retry in 853.445471ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb
15532025-01-08T18:21:47.349ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd") failed: Property not found. retry in 1.179862199s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd
15542025-01-08T18:21:47.368ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8") failed: Property not found. retry in 750.288822ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
15552025-01-08T18:21:47.528ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81") failed: Property not found. retry in 612.723656ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81
15562025-01-08T18:21:47.544ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b") failed: Property not found. retry in 545.989081ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b
15572025-01-08T18:21:47.603ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3") failed: Property not found. retry in 1.30746092s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3
15582025-01-08T18:21:47.629ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501") failed: Property not found. retry in 932.554289ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501
15592025-01-08T18:21:47.690ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a") failed: Property not found. retry in 1.256990253s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a
15602025-01-08T18:21:47.894ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63") failed: Property not found. retry in 965.174507ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63
15612025-01-08T18:21:48.004ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd") failed: Property not found. retry in 1.244297969s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd
15622025-01-08T18:21:48.022ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f") failed: Property not found. retry in 1.231984493s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f
15632025-01-08T18:21:48.073ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e") failed: Property not found. retry in 734.739188ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e
15642025-01-08T18:21:48.103ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a") failed: Property not found. retry in 737.467583ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a
15652025-01-08T18:21:48.124ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b") failed: Property not found. retry in 1.446739471s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b
15662025-01-08T18:21:48.154ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8") failed: Property not found. retry in 1.142735972s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
15672025-01-08T18:21:48.178ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81") failed: Property not found. retry in 528.856332ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81
15682025-01-08T18:21:48.216ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb") failed: Property not found. retry in 1.082212265s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb
15692025-01-08T18:21:48.311ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3") failed: Property not found. retry in 900.523471ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3
15702025-01-08T18:21:48.426ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda") failed: Property not found. retry in 1.080926529s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda
15712025-01-08T18:21:48.557ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd") failed: Property not found. retry in 1.465577639s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd
15722025-01-08T18:21:48.583ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c") failed: Property not found. retry in 867.550335ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c
15732025-01-08T18:21:48.614ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646") failed: Property not found. retry in 644.957113ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646
15742025-01-08T18:21:48.644ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501") failed: Property not found. retry in 611.738861ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501
15752025-01-08T18:21:48.674ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766") failed: Property not found. retry in 976.660921ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766
15762025-01-08T18:21:48.727ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81") failed: Property not found. retry in 1.42437128s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81
15772025-01-08T18:21:48.830ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e") failed: Property not found. retry in 966.417578ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e
15782025-01-08T18:21:48.859ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a") failed: Property not found. retry in 1.260993701s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a
15792025-01-08T18:21:48.875ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63") failed: Property not found. retry in 904.718957ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63
15802025-01-08T18:21:48.930ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3") failed: Property not found. retry in 606.653323ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3
15812025-01-08T18:21:48.962ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a") failed: Property not found. retry in 1.421646974s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a
15822025-01-08T18:21:49.225ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3") failed: Property not found. retry in 879.502278ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3
15832025-01-08T18:21:49.264ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd") failed: Property not found. retry in 1.166154788s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd
15842025-01-08T18:21:49.278ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f") failed: Property not found. retry in 680.634491ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f
15852025-01-08T18:21:49.290ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501") failed: Property not found. retry in 621.741877ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501
15862025-01-08T18:21:49.304ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646") failed: Property not found. retry in 638.270442ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646
15872025-01-08T18:21:49.318ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb") failed: Property not found. retry in 635.569588ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb
15882025-01-08T18:21:49.332ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8") failed: Property not found. retry in 930.225321ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
15892025-01-08T18:21:49.465ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c") failed: Property not found. retry in 1.172052165s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c
15902025-01-08T18:21:49.520ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda") failed: Property not found. retry in 1.496574011s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda
15912025-01-08T18:21:49.550ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3") failed: Property not found. retry in 526.712076ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3
15922025-01-08T18:21:49.585ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b") failed: Property not found. retry in 1.384732196s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b
15932025-01-08T18:21:49.665ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766") failed: Property not found. retry in 1.38462273s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766
15942025-01-08T18:21:49.796ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63") failed: Property not found. retry in 751.296508ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63
15952025-01-08T18:21:49.810ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e") failed: Property not found. retry in 1.263346867s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e
15962025-01-08T18:21:49.929ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501") failed: Property not found. retry in 1.047373567s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501
15972025-01-08T18:21:49.956ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646") failed: Property not found. retry in 1.071761167s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646
15982025-01-08T18:21:49.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb") failed: Property not found. retry in 584.085536ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb
15992025-01-08T18:21:49.983ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f") failed: Property not found. retry in 744.801393ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f
16002025-01-08T18:21:50.037ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd") failed: Property not found. retry in 878.761227ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd
16012025-01-08T18:21:50.092ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3") failed: Property not found. retry in 934.2319ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3
16022025-01-08T18:21:50.116ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3") failed: Property not found. retry in 1.174805202s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3
16032025-01-08T18:21:50.133ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a") failed: Property not found. retry in 1.377982061s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a
16042025-01-08T18:21:50.166ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81") failed: Property not found. retry in 657.268827ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81
16052025-01-08T18:21:50.275ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8") failed: Property not found. retry in 1.030222038s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
16062025-01-08T18:21:50.397ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a") failed: Property not found. retry in 1.328267378s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a
16072025-01-08T18:21:50.443ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd") failed: Property not found. retry in 1.017862309s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd
16082025-01-08T18:21:50.560ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63") failed: Property not found. retry in 1.397188787s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63
16092025-01-08T18:21:50.570ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb") failed: Property not found. retry in 1.33183595s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb
16102025-01-08T18:21:50.655ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c") failed: Property not found. retry in 1.062329835s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c
16112025-01-08T18:21:50.741ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f") failed: Property not found. retry in 907.993359ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f
16122025-01-08T18:21:50.835ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81") failed: Property not found. retry in 590.378096ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81
16132025-01-08T18:21:50.894ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService23'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::e'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::e'/> <propval type="astring" name="listen_port" value='8123'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse-admin-single"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::e]:8888'/> <propval type="astring" name="ch_address" value='[fd00:1122:3344:101::e]:9000'/> <propval type="astring" name="ch_binary" value='/opt/oxide/clickhouse/clickhouse'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
16142025-01-08T18:21:50.895ZINFOSledAgent (ServiceManager): Booting oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
16152025-01-08T18:21:50.930ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd") failed: Property not found. retry in 1.17046664s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd
16162025-01-08T18:21:50.985ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b") failed: Property not found. retry in 1.140267313s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b
16172025-01-08T18:21:50.998ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501") failed: Property not found. retry in 1.011600142s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501
16182025-01-08T18:21:51.035ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda") failed: Property not found. retry in 1.073565634s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda
16192025-01-08T18:21:51.047ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3") failed: Property not found. retry in 989.075233ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3
16202025-01-08T18:21:51.061ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646") failed: Property not found. retry in 1.022599706s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646
16212025-01-08T18:21:51.078ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766") failed: Property not found. retry in 1.079140144s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766
16222025-01-08T18:21:51.091ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e") failed: Property not found. retry in 838.397576ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e
16232025-01-08T18:21:51.168ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f") failed: Property not found. retry in 52.202465ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
16242025-01-08T18:21:51.231ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f") failed: Property not found. retry in 128.252827ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
16252025-01-08T18:21:51.312ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3") failed: Property not found. retry in 1.256298945s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3
16262025-01-08T18:21:51.330ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8") failed: Property not found. retry in 1.194341099s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
16272025-01-08T18:21:51.393ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f") failed: Property not found. retry in 283.882545ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
16282025-01-08T18:21:51.479ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81") failed: Property not found. retry in 1.132342494s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81
16292025-01-08T18:21:51.525ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd") failed: Property not found. retry in 1.396923705s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd
16302025-01-08T18:21:51.558ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a") failed: Property not found. retry in 1.438494117s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a
16312025-01-08T18:21:51.663ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f") failed: Property not found. retry in 815.160342ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f
16322025-01-08T18:21:51.689ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f") failed: Property not found. retry in 569.518636ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
16332025-01-08T18:21:51.729ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c") failed: Property not found. retry in 900.299386ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c
16342025-01-08T18:21:51.741ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a") failed: Property not found. retry in 1.028077053s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a
16352025-01-08T18:21:52.139ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 2.428529075s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
16362025-01-08T18:21:52.151ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3") failed: Property not found. retry in 873.131307ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3
16372025-01-08T18:21:52.163ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb") failed: Property not found. retry in 888.519833ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb
16382025-01-08T18:21:52.172ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda") failed: Property not found. retry in 1.461787099s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda
16392025-01-08T18:21:52.182ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b") failed: Property not found. retry in 620.432726ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b
16402025-01-08T18:21:52.193ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501") failed: Property not found. retry in 861.527485ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501
16412025-01-08T18:21:52.203ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63") failed: Property not found. retry in 1.384433568s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63
16422025-01-08T18:21:52.213ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd") failed: Property not found. retry in 1.407633465s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd
16432025-01-08T18:21:52.223ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766") failed: Property not found. retry in 589.878198ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766
16442025-01-08T18:21:52.233ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646") failed: Property not found. retry in 1.36541616s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646
16452025-01-08T18:21:52.243ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e") failed: Property not found. retry in 974.724494ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e
16462025-01-08T18:21:52.271ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f") failed: Property not found. retry in 493.060158ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
16472025-01-08T18:21:52.491ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f") failed: Property not found. retry in 1.340469016s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f
16482025-01-08T18:21:52.538ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8") failed: Property not found. retry in 819.426644ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
16492025-01-08T18:21:52.582ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3") failed: Property not found. retry in 1.387454598s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3
16502025-01-08T18:21:52.623ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81") failed: Property not found. retry in 1.254364714s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81
16512025-01-08T18:21:52.643ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c") failed: Property not found. retry in 587.356978ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c
16522025-01-08T18:21:52.778ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f") failed: Property not found. retry in 932.308283ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
16532025-01-08T18:21:52.790ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a") failed: Property not found. retry in 986.545891ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a
16542025-01-08T18:21:52.815ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b") failed: Property not found. retry in 644.065407ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b
16552025-01-08T18:21:52.829ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766") failed: Property not found. retry in 1.023124715s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766
16562025-01-08T18:21:52.936ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd") failed: Property not found. retry in 565.044735ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd
16572025-01-08T18:21:53.008ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a") failed: Property not found. retry in 615.471807ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a
16582025-01-08T18:21:53.035ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3") failed: Property not found. retry in 689.801874ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3
16592025-01-08T18:21:53.063ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb") failed: Property not found. retry in 1.42807664s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb
16602025-01-08T18:21:53.073ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501") failed: Property not found. retry in 1.150196121s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501
16612025-01-08T18:21:53.231ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e") failed: Property not found. retry in 684.548083ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e
16622025-01-08T18:21:53.243ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c") failed: Property not found. retry in 1.144208822s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c
16632025-01-08T18:21:53.370ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8") failed: Property not found. retry in 839.765013ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
16642025-01-08T18:21:53.473ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b") failed: Property not found. retry in 1.200470558s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b
16652025-01-08T18:21:53.514ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd") failed: Property not found. retry in 685.186781ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd
16662025-01-08T18:21:53.601ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63") failed: Property not found. retry in 1.032755029s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63
16672025-01-08T18:21:53.613ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646") failed: Property not found. retry in 1.034576345s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646
16682025-01-08T18:21:53.631ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd") failed: Property not found. retry in 1.150431683s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd
16692025-01-08T18:21:53.641ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a") failed: Property not found. retry in 1.288239804s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a
16702025-01-08T18:21:53.653ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda") failed: Property not found. retry in 1.46924127s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda
16712025-01-08T18:21:53.724ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f") failed: Property not found. retry in 1.364860609s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
16722025-01-08T18:21:53.739ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3") failed: Property not found. retry in 1.114374513s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3
16732025-01-08T18:21:53.793ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a") failed: Property not found. retry in 843.626498ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a
16742025-01-08T18:21:53.847ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f") failed: Property not found. retry in 1.446120856s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f
16752025-01-08T18:21:53.867ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766") failed: Property not found. retry in 954.683466ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766
16762025-01-08T18:21:53.889ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81") failed: Property not found. retry in 885.353924ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81
16772025-01-08T18:21:53.927ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e") failed: Property not found. retry in 1.435158346s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e
16782025-01-08T18:21:53.988ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3") failed: Property not found. retry in 889.729758ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3
16792025-01-08T18:21:54.232ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8") failed: Property not found. retry in 1.136402476s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
16802025-01-08T18:21:54.410ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c") failed: Property not found. retry in 550.881673ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c
16812025-01-08T18:21:54.506ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb") failed: Property not found. retry in 1.237109024s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb
16822025-01-08T18:21:54.809ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a") failed: Property not found. retry in 1.311242638s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a
16832025-01-08T18:21:54.818ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81") failed: Property not found. retry in 948.98053ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81
16842025-01-08T18:21:54.827ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b") failed: Property not found. retry in 1.440544937s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b
16852025-01-08T18:21:54.856ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd") failed: Property not found. retry in 504.968344ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd
16862025-01-08T18:21:54.882ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646") failed: Property not found. retry in 1.054429549s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646
16872025-01-08T18:21:54.910ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3") failed: Property not found. retry in 943.469743ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3
16882025-01-08T18:21:54.973ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c") failed: Property not found. retry in 1.272980812s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c
16892025-01-08T18:21:55.100ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f") failed: Property not found. retry in 872.271021ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
16902025-01-08T18:21:55.134ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda") failed: Property not found. retry in 707.25486ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda
16912025-01-08T18:21:55.311ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f") failed: Property not found. retry in 526.146564ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f
16922025-01-08T18:21:55.372ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd") failed: Property not found. retry in 1.476195133s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd
16932025-01-08T18:21:55.386ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e") failed: Property not found. retry in 881.009054ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e
16942025-01-08T18:21:55.396ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8") failed: Property not found. retry in 596.494096ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
16952025-01-08T18:21:55.756ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb") failed: Property not found. retry in 553.163922ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb
16962025-01-08T18:21:55.781ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81") failed: Property not found. retry in 1.34673799s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81
16972025-01-08T18:21:55.853ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f") failed: Property not found. retry in 1.04098012s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f
16982025-01-08T18:21:55.866ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda") failed: Property not found. retry in 651.843653ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda
16992025-01-08T18:21:55.949ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646") failed: Property not found. retry in 1.246399229s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646
17002025-01-08T18:21:55.989ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f") failed: Property not found. retry in 785.314997ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
17012025-01-08T18:21:56.004ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8") failed: Property not found. retry in 1.346255659s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
17022025-01-08T18:21:56.788ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f") failed: Property not found. retry in 965.842183ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
17032025-01-08T18:21:57.775ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f") failed: Property not found. retry in 1.141845886s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
17042025-01-08T18:21:58.932ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f") failed: Property not found. retry in 852.779175ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
17052025-01-08T18:21:59.799ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f") failed: Property not found. retry in 924.564428ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
17062025-01-08T18:22:00.737ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f") failed: Property not found. retry in 767.934189ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
17072025-01-08T18:22:01.134ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 4.049886164s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
17082025-01-08T18:22:01.517ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f") failed: Property not found. retry in 735.995511ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
17092025-01-08T18:22:01.728ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58391
17102025-01-08T18:22:01.736ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7449 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58391 req_id = 07b41ab1-20ac-4db5-bf94-d7a0a7164717 response_code = 200 uri = /baseboard
17112025-01-08T18:22:02.264ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f") failed: Property not found. retry in 925.355968ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
17122025-01-08T18:22:03.221ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_4b41895a-9eb9-4013-a351-9044e18f7fc3
17132025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_72b64c60-6adc-43c6-9811-00f42b56c8f3
17142025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_45d1606c-760e-49e1-93f5-68d2f8a0056c
17152025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_oximeter_4daad4eb-a44a-4284-aed7-ecda6bd3827a
17162025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_external_dns_3963284d-f267-448b-bfbb-a55402c7ef81
17172025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_258c2c4c-1016-4d80-b3d4-737729d8a8bb
17182025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_33bf58e9-44fc-44da-9328-86fd24a66bda
17192025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_1e710058-8813-4cc1-b005-f322f467107b
17202025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_external_dns_1c3a7513-ed19-483c-872f-3377ba80f37f
17212025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_pantry_95ba58cc-ac8e-4184-ab2a-6133ffeee501
17222025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_288f7daa-e3bd-4928-a1ba-e69d732272fd
17232025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_0dcfd88e-be05-40ed-9457-11e0fb9fdb63
17242025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_pantry_5fe6fe4f-157b-4be8-bd09-fbf2aa4b08bd
17252025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_pantry_3df2e090-e138-45b7-ae0c-befa35147766
17262025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_77527a2c-7c89-4a87-b0cc-afb1ad48e17a
17272025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_nexus_099ecd74-9dbb-4a7f-94ce-79177e3bb646
17282025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_clickhouse_e958ccc4-cfdf-45a9-9788-badac857001f
17292025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_nexus_a28f8263-3fdd-4cd1-a79d-6ef15080a80e
17302025-01-08T18:22:03.222ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_nexus_cef4a321-387e-44a1-92c9-23673073cdd8
17312025-01-08T18:22:03.232ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
17322025-01-08T18:22:03.245ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
17332025-01-08T18:22:03.249ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 27285103 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:53881 req_id = 62d7f5e5-0c4d-4a2e-8946-06cdac0bec80 response_code = 204 uri = /omicron-zones
17342025-01-08T18:22:03.253ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1415
17352025-01-08T18:22:03.284ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:790
17362025-01-08T18:22:03.286ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::b]:12221 file = sled-agent/src/rack_setup/service.rs:802
17372025-01-08T18:22:03.327ZINFOSledAgent (RSS): rack_network_config: RackNetworkConfigV2 { bfd: [], bgp: [], infra_ip_first: 10.151.1.174, infra_ip_last: 10.151.1.174, ports: [ PortConfigV2 { addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.1.174, width: 24, }, ), vlan_id: None, }, ], autoneg: false, bgp_peers: [], lldp: None, port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.1.100, rib_priority: None, vlan_id: None, }, ], switch: Switch0, tx_eq: None, uplink_port_fec: Some( None, ), uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, } file = sled-agent/src/rack_setup/service.rs:1016
17382025-01-08T18:22:03.356ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/dc521fbd-1e78-4c20-a78c-fc16c751ec23/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1078
17392025-01-08T18:22:03.487ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/dc521fbd-1e78-4c20-a78c-fc16c751ec23/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1078
17402025-01-08T18:22:03.635ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/dc521fbd-1e78-4c20-a78c-fc16c751ec23/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1078
17412025-01-08T18:22:03.851ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/dc521fbd-1e78-4c20-a78c-fc16c751ec23/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1078
17422025-01-08T18:22:03.994ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/dc521fbd-1e78-4c20-a78c-fc16c751ec23/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1078
17432025-01-08T18:22:04.122ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/dc521fbd-1e78-4c20-a78c-fc16c751ec23/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1078
17442025-01-08T18:22:04.349ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/dc521fbd-1e78-4c20-a78c-fc16c751ec23/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1078
17452025-01-08T18:22:04.519ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/dc521fbd-1e78-4c20-a78c-fc16c751ec23/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1078
17462025-01-08T18:22:05.043ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/dc521fbd-1e78-4c20-a78c-fc16c751ec23/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1078
17472025-01-08T18:22:05.218ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 8.469233914s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
17482025-01-08T18:22:10.472ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "f8d0fdb5-e645-4431-8229-85524500c0c2", "content-length": "173", "date": "Wed, 08 Jan 2025 18:22:10 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"dc521fbd-1e78-4c20-a78c-fc16c751ec23\"", request_id: "f8d0fdb5-e645-4431-8229-85524500c0c2" } file = sled-agent/src/rack_setup/service.rs:1078
17492025-01-08T18:22:11.268ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "417df71c-2f56-436c-ab78-a0f78dac286c", "content-length": "173", "date": "Wed, 08 Jan 2025 18:22:10 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"dc521fbd-1e78-4c20-a78c-fc16c751ec23\"", request_id: "417df71c-2f56-436c-ab78-a0f78dac286c" } file = sled-agent/src/rack_setup/service.rs:1078
17502025-01-08T18:22:11.902ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "d59dfa6e-9a56-4b6a-911f-64e69b51c1da", "content-length": "173", "date": "Wed, 08 Jan 2025 18:22:11 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"dc521fbd-1e78-4c20-a78c-fc16c751ec23\"", request_id: "d59dfa6e-9a56-4b6a-911f-64e69b51c1da" } file = sled-agent/src/rack_setup/service.rs:1078
17512025-01-08T18:22:12.707ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "a8bf9002-1ccf-43e8-a348-7146984ab084", "content-length": "173", "date": "Wed, 08 Jan 2025 18:22:12 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"dc521fbd-1e78-4c20-a78c-fc16c751ec23\"", request_id: "a8bf9002-1ccf-43e8-a348-7146984ab084" } file = sled-agent/src/rack_setup/service.rs:1078
17522025-01-08T18:22:15.780ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "879d65be-2985-44b0-b73e-9a94f3eee66a", "content-length": "173", "date": "Wed, 08 Jan 2025 18:22:15 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0\"", request_id: "879d65be-2985-44b0-b73e-9a94f3eee66a" } file = sled-agent/src/rack_setup/service.rs:1078
17532025-01-08T18:22:17.683ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "f696a7e5-075e-43b0-ac33-eb29d9296df1", "content-length": "173", "date": "Wed, 08 Jan 2025 18:22:17 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0\"", request_id: "f696a7e5-075e-43b0-ac33-eb29d9296df1" } file = sled-agent/src/rack_setup/service.rs:1078
17542025-01-08T18:22:18.725ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "9b89c78e-adcc-4d9d-82ee-85eb16c6a646", "content-length": "173", "date": "Wed, 08 Jan 2025 18:22:18 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0\"", request_id: "9b89c78e-adcc-4d9d-82ee-85eb16c6a646" } file = sled-agent/src/rack_setup/service.rs:1078
17552025-01-08T18:22:21.326ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "ca467097-9af0-41a5-a194-950700cd496b", "content-length": "173", "date": "Wed, 08 Jan 2025 18:22:21 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0\"", request_id: "ca467097-9af0-41a5-a194-950700cd496b" } file = sled-agent/src/rack_setup/service.rs:1078
17562025-01-08T18:22:24.543ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "37f23d04-45a3-4a2a-91ea-772e223df1aa", "content-length": "173", "date": "Wed, 08 Jan 2025 18:22:23 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0\"", request_id: "37f23d04-45a3-4a2a-91ea-772e223df1aa" } file = sled-agent/src/rack_setup/service.rs:1078
17572025-01-08T18:22:28.034ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "7d52e4d0-1425-4332-8570-a0942befe5b4", "content-length": "173", "date": "Wed, 08 Jan 2025 18:22:27 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0\"", request_id: "7d52e4d0-1425-4332-8570-a0942befe5b4" } file = sled-agent/src/rack_setup/service.rs:1078
17582025-01-08T18:22:30.578ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "6e6da197-784a-40ea-8560-aac636b9ca11", "content-length": "173", "date": "Wed, 08 Jan 2025 18:22:30 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0\"", request_id: "6e6da197-784a-40ea-8560-aac636b9ca11" } file = sled-agent/src/rack_setup/service.rs:1078
17592025-01-08T18:22:31.774ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:42167
17602025-01-08T18:22:31.774ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:42167 req_id = 90b6a279-c01a-438e-b588-bf9e730bf0d7 response_code = 200 uri = /baseboard
17612025-01-08T18:22:34.907ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "fe329627-676c-4e0d-90b5-6f19527ea5e0", "content-length": "173", "date": "Wed, 08 Jan 2025 18:22:34 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0\"", request_id: "fe329627-676c-4e0d-90b5-6f19527ea5e0" } file = sled-agent/src/rack_setup/service.rs:1078
17622025-01-08T18:22:38.662ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "9530cbb5-ad26-4382-8733-34a2bdbc89d5", "content-length": "173", "date": "Wed, 08 Jan 2025 18:22:38 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0\"", request_id: "9530cbb5-ad26-4382-8733-34a2bdbc89d5" } file = sled-agent/src/rack_setup/service.rs:1078
17632025-01-08T18:22:42.198ZINFOSledAgent (NexusNotifierTask): Successfully put SledAgentInfo to nexus file = sled-agent/src/nexus.rs:500 new_generation = 1 old_generation = 1 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
17642025-01-08T18:22:42.293ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42745
17652025-01-08T18:22:42.295ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
17662025-01-08T18:22:42.295ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
17672025-01-08T18:22:42.297ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-zones.json file = common/src/ledger.rs:177
17682025-01-08T18:22:42.299ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-zones.json file = common/src/ledger.rs:177
17692025-01-08T18:22:42.313ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
17702025-01-08T18:22:42.314ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f
17712025-01-08T18:22:42.336ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
17722025-01-08T18:22:42.336ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_0ac8ff68-15ea-4477-a206-5a7619973e85
17732025-01-08T18:22:42.357ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
17742025-01-08T18:22:42.358ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_bf665b71-2255-48d6-b1fd-c863b4c8e1af
17752025-01-08T18:22:42.380ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
17762025-01-08T18:22:42.380ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_4f399814-e4ff-4480-aa92-c4c936f90fd0
17772025-01-08T18:22:42.399ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
17782025-01-08T18:22:42.399ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_c1f40097-34cd-4b0e-8a1d-a4c6931740f6
17792025-01-08T18:22:42.418ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
17802025-01-08T18:22:42.418ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_efc412bd-0718-4381-8d37-e5a204c8c3de
17812025-01-08T18:22:42.437ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
17822025-01-08T18:22:42.437ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_91e62592-6bd8-45a2-a311-bfef0a5977f4
17832025-01-08T18:22:42.456ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
17842025-01-08T18:22:42.456ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_93dafc63-6f00-46e6-a292-a8413d960ce4
17852025-01-08T18:22:42.476ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
17862025-01-08T18:22:42.476ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_36139540-6634-4f0c-9c90-f57efa2432be
17872025-01-08T18:22:42.486ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 193056 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42745 req_id = 505f2a26-d58b-4de3-9f17-25e6ee875233 response_code = 200 uri = /inventory
17882025-01-08T18:22:43.988ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42146
17892025-01-08T18:22:43.988ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 364 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42146 req_id = 5bfbd7de-041f-4c4b-a03f-d2b067b9c845 response_code = 200 uri = /vpc-routes
17902025-01-08T18:22:44.067ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56459
17912025-01-08T18:22:44.068ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 417 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56459 req_id = 4e6c902b-ec25-45e1-9ab7-77033ab96841 response_code = 200 uri = /v2p
17922025-01-08T18:22:44.090ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57494
17932025-01-08T18:22:44.090ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 68 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57494 req_id = ddbee151-2896-4bc3-96ee-d36859a4ebfa response_code = 200 uri = /vpc-routes
17942025-01-08T18:22:44.152ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
17952025-01-08T18:22:44.152ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 779 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57494 req_id = 722ab699-ad5a-4988-87e1-3e2449f9d64f response_code = 204 uri = /eip-gateways
17962025-01-08T18:22:44.156ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
17972025-01-08T18:22:44.156ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42146 req_id = 5286e0cc-014f-465f-a238-111bcfe0c918 response_code = 204 uri = /eip-gateways
17982025-01-08T18:22:44.212ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58943
17992025-01-08T18:22:44.218ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5091 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58943 req_id = d2feeb26-c557-4c7b-8784-e65fc39541bb response_code = 200 uri = /v2p
18002025-01-08T18:22:44.450ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46772
18012025-01-08T18:22:44.450ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46772 req_id = 6415da86-b0f4-42a0-ab4e-2af842f102c0 response_code = 200 uri = /vpc-routes
18022025-01-08T18:22:44.505ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 10896 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57494 req_id = 73daf5b4-89d6-42cd-853e-9cfcb8ea6536 response_code = 204 uri = /vpc-routes
18032025-01-08T18:22:44.522ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48013
18042025-01-08T18:22:44.522ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48013 req_id = ffcf402a-ba16-48ff-aaef-fc8215a9b1f6 response_code = 200 uri = /v2p
18052025-01-08T18:22:44.566ZINFOSledAgent (PortManager): skipping delta compute for subnet file = illumos-utils/src/opte/port_manager.rs:486 new_vers = RouterVersion { router_id: 001de000-074c-4000-8000-000000000001, version: 5 } old_vers = RouterVersion { router_id: 001de000-074c-4000-8000-000000000001, version: 5 } subnet = RouterId { vni: Vni(100), kind: System }
18062025-01-08T18:22:44.567ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 717 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42146 req_id = a4901feb-1472-4acd-a896-b58041c716c1 response_code = 204 uri = /vpc-routes
18072025-01-08T18:22:44.611ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
18082025-01-08T18:22:44.611ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:46772 req_id = b17e7066-6fb9-4226-b108-bd68c8f58b4d response_code = 204 uri = /eip-gateways
18092025-01-08T18:22:44.851ZINFOSledAgent (PortManager): skipping delta compute for subnet file = illumos-utils/src/opte/port_manager.rs:486 new_vers = RouterVersion { router_id: 001de000-074c-4000-8000-000000000001, version: 5 } old_vers = RouterVersion { router_id: 001de000-074c-4000-8000-000000000001, version: 5 } subnet = RouterId { vni: Vni(100), kind: System }
18102025-01-08T18:22:44.851ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 828 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:46772 req_id = f004ef14-581a-46c9-8854-e10eb77ca5e1 response_code = 204 uri = /vpc-routes
18112025-01-08T18:22:46.679ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41648
18122025-01-08T18:22:46.681ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0041456a-da2f-4ef8-bdf7-8e18ca9d8279, kind: Service { id: 1c3a7513-ed19-483c-872f-3377ba80f37f }, name: Name("external-dns-1c3a7513-ed19-483c-872f-3377ba80f37f"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 154, 170])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ce1a1751-7375-4744-b6f0-690b02866c95, kind: Service { id: 3963284d-f267-448b-bfbb-a55402c7ef81 }, name: Name("external-dns-3963284d-f267-448b-bfbb-a55402c7ef81"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 116])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2beb6270-a9c4-41a0-b3ab-eeff6ddd3582, kind: Service { id: cef4a321-387e-44a1-92c9-23673073cdd8 }, name: Name("nexus-cef4a321-387e-44a1-92c9-23673073cdd8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 113])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bfb8fa95-f4f1-4ec3-bb77-0e8adbc6ac68, kind: Service { id: a28f8263-3fdd-4cd1-a79d-6ef15080a80e }, name: Name("nexus-a28f8263-3fdd-4cd1-a79d-6ef15080a80e"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c656ac3d-ffdc-434e-863c-8efcfbd82ab4, kind: Service { id: 099ecd74-9dbb-4a7f-94ce-79177e3bb646 }, name: Name("nexus-099ecd74-9dbb-4a7f-94ce-79177e3bb646"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 171])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
18132025-01-08T18:22:46.681ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18142025-01-08T18:22:46.681ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18152025-01-08T18:22:46.681ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
18162025-01-08T18:22:46.681ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18172025-01-08T18:22:46.681ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18182025-01-08T18:22:46.681ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18192025-01-08T18:22:46.681ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1814 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41648 req_id = 820fccf0-4862-428e-9533-69400f529cb6 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18202025-01-08T18:22:46.683ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:1088
18212025-01-08T18:22:46.688ZINFOSledAgent (RSS): Writing ledger to /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
18222025-01-08T18:22:46.688ZINFOSledAgent (RSS): Writing ledger to /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
18232025-01-08T18:22:46.874ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50943
18242025-01-08T18:22:46.874ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0041456a-da2f-4ef8-bdf7-8e18ca9d8279, kind: Service { id: 1c3a7513-ed19-483c-872f-3377ba80f37f }, name: Name("external-dns-1c3a7513-ed19-483c-872f-3377ba80f37f"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 154, 170])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ce1a1751-7375-4744-b6f0-690b02866c95, kind: Service { id: 3963284d-f267-448b-bfbb-a55402c7ef81 }, name: Name("external-dns-3963284d-f267-448b-bfbb-a55402c7ef81"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 116])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2beb6270-a9c4-41a0-b3ab-eeff6ddd3582, kind: Service { id: cef4a321-387e-44a1-92c9-23673073cdd8 }, name: Name("nexus-cef4a321-387e-44a1-92c9-23673073cdd8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 113])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bfb8fa95-f4f1-4ec3-bb77-0e8adbc6ac68, kind: Service { id: a28f8263-3fdd-4cd1-a79d-6ef15080a80e }, name: Name("nexus-a28f8263-3fdd-4cd1-a79d-6ef15080a80e"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c656ac3d-ffdc-434e-863c-8efcfbd82ab4, kind: Service { id: 099ecd74-9dbb-4a7f-94ce-79177e3bb646 }, name: Name("nexus-099ecd74-9dbb-4a7f-94ce-79177e3bb646"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 171])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
18252025-01-08T18:22:46.874ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18262025-01-08T18:22:46.874ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18272025-01-08T18:22:46.874ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
18282025-01-08T18:22:46.874ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18292025-01-08T18:22:46.874ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18302025-01-08T18:22:46.874ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18312025-01-08T18:22:46.874ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 336 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50943 req_id = 7f633b6c-8413-4cfe-8417-62645f51a43f response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18322025-01-08T18:22:46.964ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41640
18332025-01-08T18:22:46.964ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
18342025-01-08T18:22:46.964ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
18352025-01-08T18:22:46.964ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-zones.json file = common/src/ledger.rs:177
18362025-01-08T18:22:46.964ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-zones.json file = common/src/ledger.rs:177
18372025-01-08T18:22:46.977ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18382025-01-08T18:22:46.977ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f
18392025-01-08T18:22:47.002ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18402025-01-08T18:22:47.002ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_0ac8ff68-15ea-4477-a206-5a7619973e85
18412025-01-08T18:22:47.026ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18422025-01-08T18:22:47.026ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_bf665b71-2255-48d6-b1fd-c863b4c8e1af
18432025-01-08T18:22:47.050ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18442025-01-08T18:22:47.050ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_4f399814-e4ff-4480-aa92-c4c936f90fd0
18452025-01-08T18:22:47.071ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18462025-01-08T18:22:47.071ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_c1f40097-34cd-4b0e-8a1d-a4c6931740f6
18472025-01-08T18:22:47.093ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18482025-01-08T18:22:47.093ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_efc412bd-0718-4381-8d37-e5a204c8c3de
18492025-01-08T18:22:47.115ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18502025-01-08T18:22:47.115ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_91e62592-6bd8-45a2-a311-bfef0a5977f4
18512025-01-08T18:22:47.134ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18522025-01-08T18:22:47.134ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_93dafc63-6f00-46e6-a292-a8413d960ce4
18532025-01-08T18:22:47.158ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18542025-01-08T18:22:47.158ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_36139540-6634-4f0c-9c90-f57efa2432be
18552025-01-08T18:22:47.170ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 206123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41640 req_id = 0c75fba9-274c-4765-a2fc-f8d36d3ce5df response_code = 200 uri = /inventory
18562025-01-08T18:22:47.307ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44952
18572025-01-08T18:22:47.307ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0041456a-da2f-4ef8-bdf7-8e18ca9d8279, kind: Service { id: 1c3a7513-ed19-483c-872f-3377ba80f37f }, name: Name("external-dns-1c3a7513-ed19-483c-872f-3377ba80f37f"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 154, 170])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ce1a1751-7375-4744-b6f0-690b02866c95, kind: Service { id: 3963284d-f267-448b-bfbb-a55402c7ef81 }, name: Name("external-dns-3963284d-f267-448b-bfbb-a55402c7ef81"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 116])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2beb6270-a9c4-41a0-b3ab-eeff6ddd3582, kind: Service { id: cef4a321-387e-44a1-92c9-23673073cdd8 }, name: Name("nexus-cef4a321-387e-44a1-92c9-23673073cdd8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 113])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bfb8fa95-f4f1-4ec3-bb77-0e8adbc6ac68, kind: Service { id: a28f8263-3fdd-4cd1-a79d-6ef15080a80e }, name: Name("nexus-a28f8263-3fdd-4cd1-a79d-6ef15080a80e"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c656ac3d-ffdc-434e-863c-8efcfbd82ab4, kind: Service { id: 099ecd74-9dbb-4a7f-94ce-79177e3bb646 }, name: Name("nexus-099ecd74-9dbb-4a7f-94ce-79177e3bb646"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 171])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
18582025-01-08T18:22:47.307ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18592025-01-08T18:22:47.307ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18602025-01-08T18:22:47.307ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
18612025-01-08T18:22:47.307ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18622025-01-08T18:22:47.307ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18632025-01-08T18:22:47.307ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18642025-01-08T18:22:47.307ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 289 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:44952 req_id = 84a1dbd7-0d62-4197-93e3-0b531174cf88 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18652025-01-08T18:22:47.824ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55944
18662025-01-08T18:22:47.824ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0041456a-da2f-4ef8-bdf7-8e18ca9d8279, kind: Service { id: 1c3a7513-ed19-483c-872f-3377ba80f37f }, name: Name("external-dns-1c3a7513-ed19-483c-872f-3377ba80f37f"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 154, 170])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ce1a1751-7375-4744-b6f0-690b02866c95, kind: Service { id: 3963284d-f267-448b-bfbb-a55402c7ef81 }, name: Name("external-dns-3963284d-f267-448b-bfbb-a55402c7ef81"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 116])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2beb6270-a9c4-41a0-b3ab-eeff6ddd3582, kind: Service { id: cef4a321-387e-44a1-92c9-23673073cdd8 }, name: Name("nexus-cef4a321-387e-44a1-92c9-23673073cdd8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 113])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bfb8fa95-f4f1-4ec3-bb77-0e8adbc6ac68, kind: Service { id: a28f8263-3fdd-4cd1-a79d-6ef15080a80e }, name: Name("nexus-a28f8263-3fdd-4cd1-a79d-6ef15080a80e"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c656ac3d-ffdc-434e-863c-8efcfbd82ab4, kind: Service { id: 099ecd74-9dbb-4a7f-94ce-79177e3bb646 }, name: Name("nexus-099ecd74-9dbb-4a7f-94ce-79177e3bb646"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 171])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
18672025-01-08T18:22:47.824ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18682025-01-08T18:22:47.824ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18692025-01-08T18:22:47.824ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
18702025-01-08T18:22:47.824ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18712025-01-08T18:22:47.824ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18722025-01-08T18:22:47.824ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18732025-01-08T18:22:47.824ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 300 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55944 req_id = d1ab4f7e-8087-4074-993e-1928a6a52b4c response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18742025-01-08T18:22:54.857ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53614
18752025-01-08T18:22:54.857ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53614 req_id = fec39d8a-8dae-4bda-b6ac-2884c07562cb response_code = 200 uri = /vpc-routes
18762025-01-08T18:22:54.872ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
18772025-01-08T18:22:54.872ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53614 req_id = 99e26c4c-3e3e-4a05-80fc-203127036ae7 response_code = 204 uri = /eip-gateways
18782025-01-08T18:22:54.883ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 222 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53614 req_id = fe35b6be-43ca-4157-95d5-2276a243744c response_code = 204 uri = /vpc-routes
18792025-01-08T18:22:55.652ZINFOSledAgent (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:33327 remote_addr = [fd00:1122:3344:101::d]:43465 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
18802025-01-08T18:22:55.683ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 29095 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = 7a6bf381-a41b-459c-bedc-4595972b3a8f response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
18812025-01-08T18:23:01.807ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43727
18822025-01-08T18:23:01.808ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 67 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43727 req_id = d422fe56-491f-4054-b651-e41ed7bf2887 response_code = 200 uri = /baseboard
18832025-01-08T18:23:14.024ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58040
18842025-01-08T18:23:14.024ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58040 req_id = 0b28e156-b350-4f28-8ef7-182df69e35f7 response_code = 200 uri = /vpc-routes
18852025-01-08T18:23:14.100ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
18862025-01-08T18:23:14.100ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 73 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58040 req_id = 40bd846c-8121-4e61-8af4-eadf8d03495b response_code = 204 uri = /eip-gateways
18872025-01-08T18:23:14.111ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 247 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58040 req_id = 102e05b5-e19e-475f-9d9e-c3879f79c677 response_code = 204 uri = /vpc-routes
18882025-01-08T18:23:14.217ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60752
18892025-01-08T18:23:14.217ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60752 req_id = 35739e88-d412-4a3a-9a47-8ba26c7786ec response_code = 200 uri = /vpc-routes
18902025-01-08T18:23:14.263ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52589
18912025-01-08T18:23:14.264ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
18922025-01-08T18:23:14.357ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
18932025-01-08T18:23:14.443ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 180353 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:52589 req_id = e058e209-ca35-42d6-bc29-eb4c2e2fc7eb response_code = 204 uri = /switch-ports
18942025-01-08T18:23:14.444ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49402
18952025-01-08T18:23:14.444ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49157
18962025-01-08T18:23:14.451ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7619 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49157 req_id = b9677d1b-ff5f-4c3c-ae49-fb119d506ef2 response_code = 200 uri = /v2p
18972025-01-08T18:23:14.452ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7559 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49402 req_id = a2d98844-54e5-4d2c-a59f-b3848cd1cf9b response_code = 200 uri = /v2p
18982025-01-08T18:23:14.452ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8334 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52589 req_id = 0646a374-70d1-4baf-b9d9-55ecb49d2502 response_code = 200 uri = /network-bootstore-config
18992025-01-08T18:23:14.463ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
19002025-01-08T18:23:14.463ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60752 req_id = 494eaaf3-cb12-41fb-9835-2c864662c03c response_code = 204 uri = /eip-gateways
19012025-01-08T18:23:14.473ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 193 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60752 req_id = f09cbd8e-58af-413c-b66a-72babde3e2e0 response_code = 204 uri = /vpc-routes
19022025-01-08T18:23:14.473ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54444
19032025-01-08T18:23:14.474ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 39 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54444 req_id = e630177d-9cde-4486-a89b-e0d7324c1ead response_code = 200 uri = /vpc-routes
19042025-01-08T18:23:14.521ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37963
19052025-01-08T18:23:14.521ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
19062025-01-08T18:23:14.567ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
19072025-01-08T18:23:14.635ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 114016 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:37963 req_id = 8250c4ed-394c-4bd9-925b-438b602d6261 response_code = 204 uri = /switch-ports
19082025-01-08T18:23:14.635ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
19092025-01-08T18:23:14.635ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:54444 req_id = c6431acb-bee4-487e-9409-6a57238598d1 response_code = 204 uri = /eip-gateways
19102025-01-08T18:23:14.635ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 293 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37963 req_id = a849fbf4-fd19-4756-8d02-43d5c26a236e response_code = 200 uri = /network-bootstore-config
19112025-01-08T18:23:14.637ZINFOSledAgent (bootstore): Attempting to update network config with generation: 2, current_generation: 1 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-buskin-i86pc
19122025-01-08T18:23:14.638ZINFOSledAgent (bootstore): Writing ledger to /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
19132025-01-08T18:23:14.639ZINFOSledAgent (bootstore): Writing ledger to /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
19142025-01-08T18:23:14.641ZINFOSledAgent (bootstore): Broadcasting network config with generation 2 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-buskin-i86pc
19152025-01-08T18:23:14.641ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5594 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:52589 req_id = 4eb53e4b-7a74-4e16-8a73-e55894941980 response_code = 204 uri = /network-bootstore-config
19162025-01-08T18:23:14.650ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 260 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:54444 req_id = 20c9431c-6fdb-4ffc-a4b0-9deed78e62db response_code = 204 uri = /vpc-routes
19172025-01-08T18:23:14.676ZINFOSledAgent (bootstore): Attempting to update network config with generation: 3, current_generation: 2 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-buskin-i86pc
19182025-01-08T18:23:14.676ZINFOSledAgent (bootstore): Writing ledger to /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
19192025-01-08T18:23:14.676ZINFOSledAgent (bootstore): Writing ledger to /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-buskin-i86pc
19202025-01-08T18:23:14.676ZINFOSledAgent (bootstore): Broadcasting network config with generation 3 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-buskin-i86pc
19212025-01-08T18:23:14.676ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 514 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37963 req_id = 6ff031a5-c82c-4794-b3f6-8505d5a17151 response_code = 204 uri = /network-bootstore-config
19222025-01-08T18:23:14.795ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43618
19232025-01-08T18:23:14.795ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43618 req_id = de6ae0a4-3050-4348-8593-4d63b7890fab response_code = 200 uri = /v2p
19242025-01-08T18:23:14.866ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62440
19252025-01-08T18:23:14.866ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
19262025-01-08T18:23:14.902ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
19272025-01-08T18:23:14.961ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94481 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:62440 req_id = 4261ca02-c13e-42b9-b3c2-92738f47f368 response_code = 204 uri = /switch-ports
19282025-01-08T18:23:14.962ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 228 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62440 req_id = 1a94b4eb-b63d-4189-b814-ba0b33576f06 response_code = 200 uri = /network-bootstore-config
19292025-01-08T18:23:25.660ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8298 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = cdf005df-fe2f-4af8-849e-61c66cfddbe2 response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
19302025-01-08T18:23:31.840ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:53394
19312025-01-08T18:23:31.840ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:53394 req_id = bb2d329f-b7c4-4713-9fee-570186e38099 response_code = 200 uri = /baseboard
19322025-01-08T18:23:44.050ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42114
19332025-01-08T18:23:44.050ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42114 req_id = 25f253b6-376e-48b9-b176-314964dd788e response_code = 200 uri = /vpc-routes
19342025-01-08T18:23:44.100ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
19352025-01-08T18:23:44.100ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42114 req_id = 8f1bf6fc-2236-4bac-89a7-50d7c8c28048 response_code = 204 uri = /eip-gateways
19362025-01-08T18:23:44.110ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 176 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42114 req_id = 2d539ab7-0735-4313-8b3a-7d0a3e8e32c5 response_code = 204 uri = /vpc-routes
19372025-01-08T18:23:44.127ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53412
19382025-01-08T18:23:44.127ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 28 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53412 req_id = 17f04a78-9743-4853-a793-c3b769044d15 response_code = 200 uri = /vpc-routes
19392025-01-08T18:23:44.178ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
19402025-01-08T18:23:44.178ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53412 req_id = b6fac9ae-24ee-4656-a253-908d2e1f888c response_code = 204 uri = /eip-gateways
19412025-01-08T18:23:44.188ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 138 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53412 req_id = d4b5af1d-bb0d-407c-8d24-cadcc4e12d5b response_code = 204 uri = /vpc-routes
19422025-01-08T18:23:44.218ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34599
19432025-01-08T18:23:44.218ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34599 req_id = b44c65b5-933c-4e9f-8fa1-9f50eb925e0b response_code = 200 uri = /v2p
19442025-01-08T18:23:44.225ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61900
19452025-01-08T18:23:44.225ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61900 req_id = 6cb71d9d-e930-4491-b55e-ba6b8f04a0db response_code = 200 uri = /v2p
19462025-01-08T18:23:44.241ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64345
19472025-01-08T18:23:44.241ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
19482025-01-08T18:23:44.279ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
19492025-01-08T18:23:44.362ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 121277 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:64345 req_id = 3b9da71c-b915-4903-adeb-cd5210a335e2 response_code = 204 uri = /switch-ports
19502025-01-08T18:23:44.362ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64052
19512025-01-08T18:23:44.363ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
19522025-01-08T18:23:44.363ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 354 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64345 req_id = f2bff2bd-316f-4ff4-aec4-268f1ecbd434 response_code = 200 uri = /network-bootstore-config
19532025-01-08T18:23:44.407ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
19542025-01-08T18:23:44.470ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 107378 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:64052 req_id = 1b761fa3-241e-4788-9f57-9f3e1ed15e45 response_code = 204 uri = /switch-ports
19552025-01-08T18:23:44.470ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64052 req_id = dd12b84f-5426-4cc9-aff8-026ab5592e5f response_code = 200 uri = /network-bootstore-config
19562025-01-08T18:23:44.513ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57744
19572025-01-08T18:23:44.513ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57744 req_id = f0635201-6417-4c16-a8b7-bbf7145d8d1b response_code = 200 uri = /vpc-routes
19582025-01-08T18:23:44.567ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
19592025-01-08T18:23:44.567ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57744 req_id = 91eea1a7-0405-4375-953e-ef02ce5843b0 response_code = 204 uri = /eip-gateways
19602025-01-08T18:23:44.583ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64594
19612025-01-08T18:23:44.583ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64594 req_id = 2206c17d-ee26-48b3-b3a0-2f55f3a53b8a response_code = 200 uri = /v2p
19622025-01-08T18:23:44.587ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 233 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57744 req_id = fbba175f-5ce1-4874-9f61-2dc2f285f113 response_code = 204 uri = /vpc-routes
19632025-01-08T18:23:44.658ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37467
19642025-01-08T18:23:44.658ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
19652025-01-08T18:23:44.697ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
19662025-01-08T18:23:44.755ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96810 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:37467 req_id = 203e75a1-71f0-4ec4-91a2-ca887bf5e8fc response_code = 204 uri = /switch-ports
19672025-01-08T18:23:44.755ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37467 req_id = fa62554c-7e3b-43b3-88c8-a116e0ff51c0 response_code = 200 uri = /network-bootstore-config
19682025-01-08T18:23:55.663ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 11079 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = ae752c3f-7cd6-4201-8a32-a338329f917d response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
19692025-01-08T18:24:01.873ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:55948
19702025-01-08T18:24:01.873ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:55948 req_id = d0463355-a9ed-4305-bb3b-29255ecb8d11 response_code = 200 uri = /baseboard
19712025-01-08T18:24:14.063ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41794
19722025-01-08T18:24:14.063ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41794 req_id = 7e18e2a4-3905-4938-a1ba-2f2690f8cd6b response_code = 200 uri = /vpc-routes
19732025-01-08T18:24:14.132ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
19742025-01-08T18:24:14.132ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:41794 req_id = b6279955-939b-4621-ad18-139fa3f1824d response_code = 204 uri = /eip-gateways
19752025-01-08T18:24:14.143ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 208 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:41794 req_id = 94c61435-766a-4818-940d-d763fe054d3d response_code = 204 uri = /vpc-routes
19762025-01-08T18:24:14.187ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60715
19772025-01-08T18:24:14.187ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60715 req_id = 8751d5c1-9739-4cdd-b42a-274176a4bbbf response_code = 200 uri = /vpc-routes
19782025-01-08T18:24:14.245ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
19792025-01-08T18:24:14.245ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60715 req_id = 83247669-f41a-43cd-9253-efb656786e58 response_code = 204 uri = /eip-gateways
19802025-01-08T18:24:14.255ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 165 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60715 req_id = 1f0729f0-f4ac-4024-89e7-a06094262b19 response_code = 204 uri = /vpc-routes
19812025-01-08T18:24:14.324ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46949
19822025-01-08T18:24:14.324ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 138 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46949 req_id = 5792cdaa-534e-4b4b-b698-fa11aae7ec2f response_code = 200 uri = /v2p
19832025-01-08T18:24:14.358ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48732
19842025-01-08T18:24:14.358ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48732 req_id = 6db7aa94-4798-441b-8931-6c36a3563706 response_code = 200 uri = /v2p
19852025-01-08T18:24:14.384ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48873
19862025-01-08T18:24:14.384ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
19872025-01-08T18:24:14.432ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
19882025-01-08T18:24:14.494ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 109392 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:48873 req_id = ecf45f6b-ea0c-4b3e-94dd-53bef1968451 response_code = 204 uri = /switch-ports
19892025-01-08T18:24:14.494ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52679
19902025-01-08T18:24:14.494ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 275 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48873 req_id = 429b950c-492c-444a-b406-7b496ead2c96 response_code = 200 uri = /network-bootstore-config
19912025-01-08T18:24:14.494ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
19922025-01-08T18:24:14.534ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
19932025-01-08T18:24:14.590ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95809 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:52679 req_id = 1f703c6f-92d1-4fb9-9a49-c452aba18134 response_code = 204 uri = /switch-ports
19942025-01-08T18:24:14.591ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 160 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52679 req_id = ea4dff47-7fd8-4404-8807-ef3b33bbe23f response_code = 200 uri = /network-bootstore-config
19952025-01-08T18:24:14.616ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54479
19962025-01-08T18:24:14.616ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54479 req_id = b11633f8-0b83-45f2-9f71-b231be4fd301 response_code = 200 uri = /vpc-routes
19972025-01-08T18:24:14.660ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
19982025-01-08T18:24:14.660ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 80 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:54479 req_id = 8f1cbfd3-5d0c-4d36-bfc1-411608889c2c response_code = 204 uri = /eip-gateways
19992025-01-08T18:24:14.672ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 344 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:54479 req_id = 469049db-5aaa-47cc-9122-1945ee03b233 response_code = 204 uri = /vpc-routes
20002025-01-08T18:24:14.849ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54696
20012025-01-08T18:24:14.849ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 150 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54696 req_id = 88b6ed1c-c884-4e48-8726-16f784e02c63 response_code = 200 uri = /v2p
20022025-01-08T18:24:14.887ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50489
20032025-01-08T18:24:14.887ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
20042025-01-08T18:24:14.928ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
20052025-01-08T18:24:14.984ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96265 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:50489 req_id = 27331ba7-1866-4d14-9227-b23dd5c6c413 response_code = 204 uri = /switch-ports
20062025-01-08T18:24:14.987ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 170 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50489 req_id = e55bceec-09b3-4d82-9e23-a56b54a65eb0 response_code = 200 uri = /network-bootstore-config
20072025-01-08T18:24:25.660ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8068 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = bbac7413-79be-40d4-bb30-2f49d747d295 response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
20082025-01-08T18:24:31.906ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:39855
20092025-01-08T18:24:31.906ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 52 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:39855 req_id = c2c87c2b-f53a-4d9a-b04c-d0f30fb58cd9 response_code = 200 uri = /baseboard
20102025-01-08T18:24:44.050ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54520
20112025-01-08T18:24:44.050ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 52 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54520 req_id = 7fddca7b-2897-4c29-b686-297bc502d122 response_code = 200 uri = /vpc-routes
20122025-01-08T18:24:44.066ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
20132025-01-08T18:24:44.066ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 71 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:54520 req_id = bd987f91-fb90-41d2-a7db-9b2fa7112c60 response_code = 204 uri = /eip-gateways
20142025-01-08T18:24:44.076ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 155 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:54520 req_id = fccbb050-7782-4e8f-a155-fd8c791b146f response_code = 204 uri = /vpc-routes
20152025-01-08T18:24:44.077ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57836
20162025-01-08T18:24:44.077ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 19 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57836 req_id = e6259f2a-0f29-45e5-958b-147fb7c0fcb9 response_code = 200 uri = /vpc-routes
20172025-01-08T18:24:44.092ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
20182025-01-08T18:24:44.092ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57836 req_id = b4ce6754-e759-4c4d-8048-d0d58d9828c7 response_code = 204 uri = /eip-gateways
20192025-01-08T18:24:44.100ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 120 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57836 req_id = e81bfaff-018d-469e-b91b-08c20cde3d59 response_code = 204 uri = /vpc-routes
20202025-01-08T18:24:44.116ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62878
20212025-01-08T18:24:44.116ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
20222025-01-08T18:24:44.159ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
20232025-01-08T18:24:44.221ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104645 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:62878 req_id = 7b4711b0-5392-4a96-822b-e07244b0ce66 response_code = 204 uri = /switch-ports
20242025-01-08T18:24:44.221ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42582
20252025-01-08T18:24:44.221ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35648
20262025-01-08T18:24:44.221ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34325
20272025-01-08T18:24:44.221ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 325 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42582 req_id = 5088a0b0-e670-4403-8331-3bfe228dcf5f response_code = 200 uri = /v2p
20282025-01-08T18:24:44.221ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 351 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62878 req_id = 2c1a9bb0-2d6c-45ba-b0ab-a2d390aa5307 response_code = 200 uri = /network-bootstore-config
20292025-01-08T18:24:44.221ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
20302025-01-08T18:24:44.221ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 379 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34325 req_id = 94163634-0b24-4e55-8ef9-e34b824eca0d response_code = 200 uri = /v2p
20312025-01-08T18:24:44.259ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
20322025-01-08T18:24:44.330ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108492 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:35648 req_id = e173d96c-5ae7-4e11-8219-c88f50a3ee4b response_code = 204 uri = /switch-ports
20332025-01-08T18:24:44.330ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 238 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35648 req_id = ae428550-f15a-42f8-ad6b-c3f3dfeae982 response_code = 200 uri = /network-bootstore-config
20342025-01-08T18:24:44.489ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61044
20352025-01-08T18:24:44.489ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61044 req_id = c4224ff4-2272-4a3d-b744-167eb899ab02 response_code = 200 uri = /vpc-routes
20362025-01-08T18:24:44.509ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
20372025-01-08T18:24:44.509ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61044 req_id = 4aa8e0f2-151a-402e-beac-56290ff1e8bd response_code = 204 uri = /eip-gateways
20382025-01-08T18:24:44.519ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 180 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61044 req_id = 5b4f42d3-9345-4c16-94c9-907f88e387ec response_code = 204 uri = /vpc-routes
20392025-01-08T18:24:44.560ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50006
20402025-01-08T18:24:44.560ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50006 req_id = 11315807-5e30-4988-91f3-0597c9c26537 response_code = 200 uri = /v2p
20412025-01-08T18:24:44.616ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:32783
20422025-01-08T18:24:44.616ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
20432025-01-08T18:24:44.654ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
20442025-01-08T18:24:44.720ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104228 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:32783 req_id = 9ed7c181-8701-4010-b920-c740f2a99fc4 response_code = 204 uri = /switch-ports
20452025-01-08T18:24:44.721ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 122 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:32783 req_id = 67cbbc01-3a6f-440e-9c47-ae6cf7e11e3f response_code = 200 uri = /network-bootstore-config
20462025-01-08T18:24:55.660ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7896 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = 38f7b47b-f80e-4b23-b672-52d684488204 response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
20472025-01-08T18:25:01.940ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:46000
20482025-01-08T18:25:01.940ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:46000 req_id = b5724e0a-1cd8-477c-b511-78f6abecff1c response_code = 200 uri = /baseboard
20492025-01-08T18:25:10.335ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
20502025-01-08T18:25:14.177ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58058
20512025-01-08T18:25:14.177ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58058 req_id = 841f1cd7-2a0e-4d6c-bac5-0e5d43672f6b response_code = 200 uri = /vpc-routes
20522025-01-08T18:25:14.222ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
20532025-01-08T18:25:14.222ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58058 req_id = 97afd6a6-5edb-4346-85ff-38fa3bc7b30c response_code = 204 uri = /eip-gateways
20542025-01-08T18:25:14.225ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35059
20552025-01-08T18:25:14.226ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 30 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35059 req_id = 3b45e554-ecab-46cf-84c3-44e34be8b715 response_code = 200 uri = /vpc-routes
20562025-01-08T18:25:14.232ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 150 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58058 req_id = cdd292e8-805e-477c-87dd-cc031da6f9a5 response_code = 204 uri = /vpc-routes
20572025-01-08T18:25:14.269ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
20582025-01-08T18:25:14.269ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 27 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35059 req_id = c0bb7c3e-4c01-4fcf-9e7c-41a3e5045153 response_code = 204 uri = /eip-gateways
20592025-01-08T18:25:14.279ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47893
20602025-01-08T18:25:14.279ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47893 req_id = 1ce57381-df27-48bf-8012-a9e263634bad response_code = 200 uri = /v2p
20612025-01-08T18:25:14.351ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63263
20622025-01-08T18:25:14.352ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
20632025-01-08T18:25:14.402ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
20642025-01-08T18:25:14.480ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 128043 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:63263 req_id = e44d48a1-5623-4774-a2ed-037f6c0ebd95 response_code = 204 uri = /switch-ports
20652025-01-08T18:25:14.480ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51576
20662025-01-08T18:25:14.480ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48858
20672025-01-08T18:25:14.480ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 298 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35059 req_id = 11a88fad-f0c7-49fa-bacc-8e390367bc82 response_code = 204 uri = /vpc-routes
20682025-01-08T18:25:14.480ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 332 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63263 req_id = d520ff01-b1fc-4b76-a474-4bcd35aeb1e9 response_code = 200 uri = /network-bootstore-config
20692025-01-08T18:25:14.480ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 271 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51576 req_id = d9372db4-848b-4d6b-ab27-2405951773ec response_code = 200 uri = /v2p
20702025-01-08T18:25:14.480ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
20712025-01-08T18:25:14.520ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
20722025-01-08T18:25:14.574ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94276 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:48858 req_id = 3e36d5ef-9732-4635-9c4e-8b17f9b2845e response_code = 204 uri = /switch-ports
20732025-01-08T18:25:14.575ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48858 req_id = c42c155d-3942-4b68-b2b6-2885eb814e16 response_code = 200 uri = /network-bootstore-config
20742025-01-08T18:25:14.595ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61652
20752025-01-08T18:25:14.595ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 68 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61652 req_id = 64e533d7-90b4-4a14-bbef-1049c9cf2561 response_code = 200 uri = /vpc-routes
20762025-01-08T18:25:14.636ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
20772025-01-08T18:25:14.636ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 71 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61652 req_id = b93fb09b-58eb-4925-ba03-b3756b0a783b response_code = 204 uri = /eip-gateways
20782025-01-08T18:25:14.648ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 294 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61652 req_id = 7d4cd2c4-96e6-4fb2-b686-bb67c6620e08 response_code = 204 uri = /vpc-routes
20792025-01-08T18:25:14.768ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37139
20802025-01-08T18:25:14.768ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37139 req_id = 96a15c2e-66e3-4a6a-8a7d-f9dc641b24f3 response_code = 200 uri = /v2p
20812025-01-08T18:25:14.940ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58442
20822025-01-08T18:25:14.940ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
20832025-01-08T18:25:14.978ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
20842025-01-08T18:25:15.032ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92159 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:58442 req_id = a7185516-4bf9-4608-8058-6aa738119d47 response_code = 204 uri = /switch-ports
20852025-01-08T18:25:15.032ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 213 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58442 req_id = 5cb8fd3e-924a-4be4-8e55-6a6749dfc624 response_code = 200 uri = /network-bootstore-config
20862025-01-08T18:25:25.660ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8116 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = 4a67ceab-7da0-4ff2-9ee8-4ad5accbbba1 response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
20872025-01-08T18:25:31.975ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:49593
20882025-01-08T18:25:31.975ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 77 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:49593 req_id = c74a4dc9-3106-473a-b329-2b8d7b621ca5 response_code = 200 uri = /baseboard
20892025-01-08T18:25:44.086ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56608
20902025-01-08T18:25:44.086ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56608 req_id = be21ad20-8a36-4261-9f26-2bf8b7cc1760 response_code = 200 uri = /vpc-routes
20912025-01-08T18:25:44.123ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56690
20922025-01-08T18:25:44.123ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 158 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56690 req_id = ad12c9fc-f56f-4708-a412-5de174a606b0 response_code = 200 uri = /v2p
20932025-01-08T18:25:44.136ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33412
20942025-01-08T18:25:44.137ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33412 req_id = ea1e8251-7cc9-4c91-a3f5-3e0742fdf650 response_code = 200 uri = /vpc-routes
20952025-01-08T18:25:44.151ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
20962025-01-08T18:25:44.151ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:33412 req_id = 1a5645ed-ceea-4511-8166-59adf5437941 response_code = 204 uri = /eip-gateways
20972025-01-08T18:25:44.167ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62446
20982025-01-08T18:25:44.167ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
20992025-01-08T18:25:44.207ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
21002025-01-08T18:25:44.265ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98212 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:62446 req_id = db9e6c49-c371-4f96-bfbe-26f16df475d7 response_code = 204 uri = /switch-ports
21012025-01-08T18:25:44.265ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50991
21022025-01-08T18:25:44.265ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41005
21032025-01-08T18:25:44.265ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
21042025-01-08T18:25:44.266ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 298 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56608 req_id = 478a655e-6395-4d71-a9b5-d9fb31932203 response_code = 204 uri = /eip-gateways
21052025-01-08T18:25:44.266ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 554 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:33412 req_id = ae49f165-ac9e-4e7d-9aa6-71600c6cfbfe response_code = 204 uri = /vpc-routes
21062025-01-08T18:25:44.266ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 361 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62446 req_id = ab44809c-edb0-4721-8f22-d3197407a042 response_code = 200 uri = /network-bootstore-config
21072025-01-08T18:25:44.266ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
21082025-01-08T18:25:44.266ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 321 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50991 req_id = 7992686c-2c57-4c43-a2ab-8bfbb824af57 response_code = 200 uri = /v2p
21092025-01-08T18:25:44.276ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 353 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56608 req_id = 43ac2821-0251-486b-903a-20a75f77ca7c response_code = 204 uri = /vpc-routes
21102025-01-08T18:25:44.316ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
21112025-01-08T18:25:44.385ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 119298 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:41005 req_id = 8c0c553d-f263-4de8-be51-7cef215aa775 response_code = 204 uri = /switch-ports
21122025-01-08T18:25:44.385ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 157 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41005 req_id = 87f3d380-0960-4239-8f69-20898cf4bf9f response_code = 200 uri = /network-bootstore-config
21132025-01-08T18:25:44.437ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50586
21142025-01-08T18:25:44.437ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50586 req_id = 06b0794d-8e2f-41f8-ae25-5a4aed5ff808 response_code = 200 uri = /vpc-routes
21152025-01-08T18:25:44.476ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43837
21162025-01-08T18:25:44.476ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43837 req_id = 81172847-15be-44da-82da-ca51b7da2205 response_code = 200 uri = /v2p
21172025-01-08T18:25:44.478ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
21182025-01-08T18:25:44.478ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 54 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:50586 req_id = a388ebff-d7c5-4f74-ba00-77b4253547a3 response_code = 204 uri = /eip-gateways
21192025-01-08T18:25:44.487ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 171 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:50586 req_id = 2631fed8-9a75-4096-b046-c0ba555bbbec response_code = 204 uri = /vpc-routes
21202025-01-08T18:25:44.521ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42569
21212025-01-08T18:25:44.521ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
21222025-01-08T18:25:44.557ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
21232025-01-08T18:25:44.614ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92751 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:42569 req_id = 768fbabe-3c68-475b-a2e7-390f7d282387 response_code = 204 uri = /switch-ports
21242025-01-08T18:25:44.615ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 193 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42569 req_id = 7d1b809f-a12e-435b-a738-b73940357535 response_code = 200 uri = /network-bootstore-config
21252025-01-08T18:25:55.659ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7606 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = f3d9b000-a179-4b73-9ba5-4463f7ef138c response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
21262025-01-08T18:26:02.010ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43271
21272025-01-08T18:26:02.010ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43271 req_id = 04dce09e-f093-4ac9-8299-6e9e17ff7d91 response_code = 200 uri = /baseboard
21282025-01-08T18:26:14.092ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37315
21292025-01-08T18:26:14.092ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37315 req_id = fa2b6a73-6b8a-42ed-afb7-66cec518bb35 response_code = 200 uri = /vpc-routes
21302025-01-08T18:26:14.109ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
21312025-01-08T18:26:14.109ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:37315 req_id = 63e82d43-fb0f-4d60-b463-e833d5b1083f response_code = 204 uri = /eip-gateways
21322025-01-08T18:26:14.118ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:37315 req_id = 85eed118-bf4c-4a88-8a84-b3a6566767af response_code = 204 uri = /vpc-routes
21332025-01-08T18:26:14.157ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57066
21342025-01-08T18:26:14.157ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 29 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57066 req_id = ebbab09e-27e7-42fa-abbd-1374b11fb5d3 response_code = 200 uri = /vpc-routes
21352025-01-08T18:26:14.160ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47194
21362025-01-08T18:26:14.160ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47194 req_id = 22643a5f-aa31-4daf-b538-1837cab07c46 response_code = 200 uri = /v2p
21372025-01-08T18:26:14.174ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
21382025-01-08T18:26:14.174ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:57066 req_id = 19e3256c-cf08-4d94-9d6c-7f157dc16429 response_code = 204 uri = /eip-gateways
21392025-01-08T18:26:14.183ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 186 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:57066 req_id = 466ba71c-ec5e-4746-a1c6-621b0f8883ac response_code = 204 uri = /vpc-routes
21402025-01-08T18:26:14.205ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40243
21412025-01-08T18:26:14.205ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40243 req_id = 3eb3f7a3-e7d3-45a2-a974-8a6a2b9a4636 response_code = 200 uri = /v2p
21422025-01-08T18:26:14.385ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48826
21432025-01-08T18:26:14.385ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
21442025-01-08T18:26:14.434ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
21452025-01-08T18:26:14.504ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 118262 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:48826 req_id = 67ecf583-00c0-47d0-861c-acd7a56967b6 response_code = 204 uri = /switch-ports
21462025-01-08T18:26:14.504ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58784
21472025-01-08T18:26:14.504ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
21482025-01-08T18:26:14.538ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
21492025-01-08T18:26:14.591ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87343 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:58784 req_id = 7b8cd5ab-0d6d-4ca1-94fe-43aa4987e862 response_code = 204 uri = /switch-ports
21502025-01-08T18:26:14.591ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61795
21512025-01-08T18:26:14.592ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 163 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61795 req_id = 2f9de56e-f3ea-4882-90b1-692cb738736c response_code = 200 uri = /vpc-routes
21522025-01-08T18:26:14.592ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 259 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58784 req_id = c8655bb6-e21d-44fa-a6b2-d64ff46bf3af response_code = 200 uri = /network-bootstore-config
21532025-01-08T18:26:14.592ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 319 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48826 req_id = 067964b8-b799-445a-8402-f3526d1d7ce8 response_code = 200 uri = /network-bootstore-config
21542025-01-08T18:26:14.607ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
21552025-01-08T18:26:14.607ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61795 req_id = 5faec917-a9b0-4f99-a1d1-97387a7f6489 response_code = 204 uri = /eip-gateways
21562025-01-08T18:26:14.616ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61795 req_id = 91c01a58-dcb0-4c6e-b591-af63d29cc46b response_code = 204 uri = /vpc-routes
21572025-01-08T18:26:14.696ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:32915
21582025-01-08T18:26:14.696ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:32915 req_id = d8013677-31d4-4f60-a976-c6e711044774 response_code = 200 uri = /v2p
21592025-01-08T18:26:14.705ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39154
21602025-01-08T18:26:14.705ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
21612025-01-08T18:26:14.741ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
21622025-01-08T18:26:14.799ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93729 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:39154 req_id = 44006aca-aaad-42c2-858f-d18779c6f8d9 response_code = 204 uri = /switch-ports
21632025-01-08T18:26:14.799ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39154 req_id = 795c1139-636d-4181-bff7-06b1dd8022ca response_code = 200 uri = /network-bootstore-config
21642025-01-08T18:26:25.660ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8008 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = fdece193-7c5f-481a-812f-3897fe176575 response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
21652025-01-08T18:26:32.043ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:46169
21662025-01-08T18:26:32.043ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 52 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:46169 req_id = e8ec1604-ed5d-4b28-97db-2dc1d812301c response_code = 200 uri = /baseboard
21672025-01-08T18:26:43.969ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42115
21682025-01-08T18:26:43.969ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42115 req_id = bd6110b9-5d3e-4f03-8034-c543e4aa2f95 response_code = 200 uri = /vpc-routes
21692025-01-08T18:26:43.983ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
21702025-01-08T18:26:43.983ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42115 req_id = b9408d58-ba7c-48f4-8b4d-fcba31c33bc3 response_code = 204 uri = /eip-gateways
21712025-01-08T18:26:43.992ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 177 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42115 req_id = b1b4d913-e758-4d6a-8042-8ea0132a902a response_code = 204 uri = /vpc-routes
21722025-01-08T18:26:44.030ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48927
21732025-01-08T18:26:44.030ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 29 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48927 req_id = 6d6a76ec-04f8-473b-aa1d-ae6b342cbc67 response_code = 200 uri = /vpc-routes
21742025-01-08T18:26:44.045ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
21752025-01-08T18:26:44.046ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48927 req_id = 4f0ffa16-7913-46b0-835a-a9ef1cd31f02 response_code = 204 uri = /eip-gateways
21762025-01-08T18:26:44.054ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 150 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48927 req_id = ab4d42d5-e1c8-4722-9e0d-750271332a5e response_code = 204 uri = /vpc-routes
21772025-01-08T18:26:44.102ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56718
21782025-01-08T18:26:44.102ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 120 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56718 req_id = bbd702b4-2e3b-4f91-8454-2ab32d7b5012 response_code = 200 uri = /v2p
21792025-01-08T18:26:44.102ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35973
21802025-01-08T18:26:44.102ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35973 req_id = 1f7a4700-8e22-4f15-8745-3f2f00b20d67 response_code = 200 uri = /v2p
21812025-01-08T18:26:44.135ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41684
21822025-01-08T18:26:44.135ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
21832025-01-08T18:26:44.172ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
21842025-01-08T18:26:44.225ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89650 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:41684 req_id = efb53495-31be-4202-a13d-ceeddbd21496 response_code = 204 uri = /switch-ports
21852025-01-08T18:26:44.225ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38095
21862025-01-08T18:26:44.225ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
21872025-01-08T18:26:44.257ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
21882025-01-08T18:26:44.328ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102681 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:38095 req_id = 0dbcd28d-b7fc-4f82-b31c-2ae89df59900 response_code = 204 uri = /switch-ports
21892025-01-08T18:26:44.328ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41684 req_id = b4aafd23-987f-4f2a-b8f4-ec34a3609499 response_code = 200 uri = /network-bootstore-config
21902025-01-08T18:26:44.329ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 159 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38095 req_id = d50a1765-eaee-4b96-9430-f3d83a335b76 response_code = 200 uri = /network-bootstore-config
21912025-01-08T18:26:44.512ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45336
21922025-01-08T18:26:44.513ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 54 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45336 req_id = 5b31513c-ce44-4f2e-bc91-2b04afc05c9d response_code = 200 uri = /vpc-routes
21932025-01-08T18:26:44.559ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
21942025-01-08T18:26:44.559ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45336 req_id = 5b2c2c52-c701-4752-a55a-fb0c2405dea9 response_code = 204 uri = /eip-gateways
21952025-01-08T18:26:44.569ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45336 req_id = 7e8721ec-bb28-415d-8f19-1f4518c2b9ea response_code = 204 uri = /vpc-routes
21962025-01-08T18:26:44.581ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38741
21972025-01-08T18:26:44.581ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38741 req_id = 94d35962-e508-4cf3-a114-826e2cefdcb6 response_code = 200 uri = /v2p
21982025-01-08T18:26:44.620ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37452
21992025-01-08T18:26:44.620ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
22002025-01-08T18:26:44.659ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
22012025-01-08T18:26:44.715ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95317 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:37452 req_id = 77cf7cbc-593c-45ae-90a1-ddf55389f1df response_code = 204 uri = /switch-ports
22022025-01-08T18:26:44.716ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 221 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37452 req_id = 93e0c715-9522-4d26-a2b1-7a3a19871a08 response_code = 200 uri = /network-bootstore-config
22032025-01-08T18:26:55.659ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7225 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = 116a9688-86b4-473b-b6b4-e1630b2d0ca0 response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
22042025-01-08T18:27:02.078ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:63686
22052025-01-08T18:27:02.078ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:63686 req_id = f2c3d112-a6c3-483a-894b-cbc7f8006b3a response_code = 200 uri = /baseboard
22062025-01-08T18:27:14.104ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51743
22072025-01-08T18:27:14.104ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51743 req_id = 97e64557-93c0-4fef-b9a7-28d247e8fb85 response_code = 200 uri = /vpc-routes
22082025-01-08T18:27:14.119ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
22092025-01-08T18:27:14.119ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 43 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:51743 req_id = 87c1badf-92ad-4750-81cb-1ac95a36406b response_code = 204 uri = /eip-gateways
22102025-01-08T18:27:14.234ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61883
22112025-01-08T18:27:14.234ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61883 req_id = 2555d709-1e4b-4d10-a174-5cd4f8d8c0c4 response_code = 200 uri = /v2p
22122025-01-08T18:27:14.236ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42621
22132025-01-08T18:27:14.236ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 80 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42621 req_id = e869cc7a-6021-4e7d-993b-e9ac945102ee response_code = 200 uri = /v2p
22142025-01-08T18:27:14.264ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43643
22152025-01-08T18:27:14.264ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 19 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43643 req_id = ea0253eb-9582-4c33-a156-5b26e8cd364e response_code = 200 uri = /vpc-routes
22162025-01-08T18:27:14.279ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
22172025-01-08T18:27:14.279ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 34 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:43643 req_id = 3a75a298-0a76-40ed-a828-74f07f5b89f0 response_code = 204 uri = /eip-gateways
22182025-01-08T18:27:14.294ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 170 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:43643 req_id = e05118c3-6edc-43a5-b44b-3ba6a75b0c3c response_code = 204 uri = /vpc-routes
22192025-01-08T18:27:14.422ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:65533
22202025-01-08T18:27:14.422ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0041456a-da2f-4ef8-bdf7-8e18ca9d8279, kind: Service { id: 1c3a7513-ed19-483c-872f-3377ba80f37f }, name: Name("external-dns-1c3a7513-ed19-483c-872f-3377ba80f37f"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 154, 170])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ce1a1751-7375-4744-b6f0-690b02866c95, kind: Service { id: 3963284d-f267-448b-bfbb-a55402c7ef81 }, name: Name("external-dns-3963284d-f267-448b-bfbb-a55402c7ef81"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 116])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2beb6270-a9c4-41a0-b3ab-eeff6ddd3582, kind: Service { id: cef4a321-387e-44a1-92c9-23673073cdd8 }, name: Name("nexus-cef4a321-387e-44a1-92c9-23673073cdd8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 113])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bfb8fa95-f4f1-4ec3-bb77-0e8adbc6ac68, kind: Service { id: a28f8263-3fdd-4cd1-a79d-6ef15080a80e }, name: Name("nexus-a28f8263-3fdd-4cd1-a79d-6ef15080a80e"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c656ac3d-ffdc-434e-863c-8efcfbd82ab4, kind: Service { id: 099ecd74-9dbb-4a7f-94ce-79177e3bb646 }, name: Name("nexus-099ecd74-9dbb-4a7f-94ce-79177e3bb646"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 171])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
22212025-01-08T18:27:14.422ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
22222025-01-08T18:27:14.422ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22232025-01-08T18:27:14.422ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
22242025-01-08T18:27:14.422ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22252025-01-08T18:27:14.422ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22262025-01-08T18:27:14.422ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
22272025-01-08T18:27:14.422ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 270 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:65533 req_id = 41fa434c-180a-4b3e-80bc-8a5f79ee541b response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
22282025-01-08T18:27:14.425ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 150 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:51743 req_id = dacfc207-0560-4cc0-b37c-ffc59fc2b6b0 response_code = 204 uri = /vpc-routes
22292025-01-08T18:27:14.446ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33978
22302025-01-08T18:27:14.446ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0041456a-da2f-4ef8-bdf7-8e18ca9d8279, kind: Service { id: 1c3a7513-ed19-483c-872f-3377ba80f37f }, name: Name("external-dns-1c3a7513-ed19-483c-872f-3377ba80f37f"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 154, 170])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ce1a1751-7375-4744-b6f0-690b02866c95, kind: Service { id: 3963284d-f267-448b-bfbb-a55402c7ef81 }, name: Name("external-dns-3963284d-f267-448b-bfbb-a55402c7ef81"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 116])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2beb6270-a9c4-41a0-b3ab-eeff6ddd3582, kind: Service { id: cef4a321-387e-44a1-92c9-23673073cdd8 }, name: Name("nexus-cef4a321-387e-44a1-92c9-23673073cdd8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 113])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bfb8fa95-f4f1-4ec3-bb77-0e8adbc6ac68, kind: Service { id: a28f8263-3fdd-4cd1-a79d-6ef15080a80e }, name: Name("nexus-a28f8263-3fdd-4cd1-a79d-6ef15080a80e"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c656ac3d-ffdc-434e-863c-8efcfbd82ab4, kind: Service { id: 099ecd74-9dbb-4a7f-94ce-79177e3bb646 }, name: Name("nexus-099ecd74-9dbb-4a7f-94ce-79177e3bb646"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 171])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
22312025-01-08T18:27:14.446ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
22322025-01-08T18:27:14.446ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22332025-01-08T18:27:14.446ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
22342025-01-08T18:27:14.446ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22352025-01-08T18:27:14.446ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22362025-01-08T18:27:14.446ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
22372025-01-08T18:27:14.446ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 172 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:33978 req_id = 504864d6-9a6c-4f8b-8cd8-221a2526f3f9 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
22382025-01-08T18:27:14.474ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55527
22392025-01-08T18:27:14.474ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
22402025-01-08T18:27:14.514ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
22412025-01-08T18:27:14.572ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97583 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:55527 req_id = af439990-a293-457d-88e2-021d93a04ece response_code = 204 uri = /switch-ports
22422025-01-08T18:27:14.572ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38378
22432025-01-08T18:27:14.572ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56450
22442025-01-08T18:27:14.572ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 308 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55527 req_id = 8f83a773-ac89-4eef-92f7-fcc19aae2412 response_code = 200 uri = /network-bootstore-config
22452025-01-08T18:27:14.572ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
22462025-01-08T18:27:14.572ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 170 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56450 req_id = 95d8ce0a-78a7-4bbb-acdd-71279b146275 response_code = 200 uri = /vpc-routes
22472025-01-08T18:27:14.619ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
22482025-01-08T18:27:14.625ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
22492025-01-08T18:27:14.625ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 213 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:56450 req_id = ba389987-04ec-4793-b3ea-dafe8286d36f response_code = 204 uri = /eip-gateways
22502025-01-08T18:27:14.634ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 327 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:56450 req_id = 59f81341-20d1-482e-9063-32827cfb3d18 response_code = 204 uri = /vpc-routes
22512025-01-08T18:27:14.681ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108859 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:38378 req_id = e2e4f74f-d44e-4086-9cde-a1b94a7b56d1 response_code = 204 uri = /switch-ports
22522025-01-08T18:27:14.682ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 149 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38378 req_id = 11b5d516-5f4c-44be-ae6a-527630bcad6e response_code = 200 uri = /network-bootstore-config
22532025-01-08T18:27:14.733ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47014
22542025-01-08T18:27:14.733ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
22552025-01-08T18:27:14.773ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51635
22562025-01-08T18:27:14.773ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 163 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51635 req_id = db15a109-eff1-4923-8cd6-438a2798cca2 response_code = 200 uri = /v2p
22572025-01-08T18:27:14.774ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
22582025-01-08T18:27:14.834ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101293 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:47014 req_id = 17851671-0427-4a98-b963-d213733a4a4c response_code = 204 uri = /switch-ports
22592025-01-08T18:27:14.835ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 150 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47014 req_id = 5b9b3d9a-ccd1-4a89-bcbd-f056139821c0 response_code = 200 uri = /network-bootstore-config
22602025-01-08T18:27:14.951ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57068
22612025-01-08T18:27:14.951ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0041456a-da2f-4ef8-bdf7-8e18ca9d8279, kind: Service { id: 1c3a7513-ed19-483c-872f-3377ba80f37f }, name: Name("external-dns-1c3a7513-ed19-483c-872f-3377ba80f37f"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 154, 170])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ce1a1751-7375-4744-b6f0-690b02866c95, kind: Service { id: 3963284d-f267-448b-bfbb-a55402c7ef81 }, name: Name("external-dns-3963284d-f267-448b-bfbb-a55402c7ef81"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 116])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2beb6270-a9c4-41a0-b3ab-eeff6ddd3582, kind: Service { id: cef4a321-387e-44a1-92c9-23673073cdd8 }, name: Name("nexus-cef4a321-387e-44a1-92c9-23673073cdd8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 113])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bfb8fa95-f4f1-4ec3-bb77-0e8adbc6ac68, kind: Service { id: a28f8263-3fdd-4cd1-a79d-6ef15080a80e }, name: Name("nexus-a28f8263-3fdd-4cd1-a79d-6ef15080a80e"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c656ac3d-ffdc-434e-863c-8efcfbd82ab4, kind: Service { id: 099ecd74-9dbb-4a7f-94ce-79177e3bb646 }, name: Name("nexus-099ecd74-9dbb-4a7f-94ce-79177e3bb646"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 171])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
22622025-01-08T18:27:14.952ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
22632025-01-08T18:27:14.952ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22642025-01-08T18:27:14.952ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
22652025-01-08T18:27:14.952ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22662025-01-08T18:27:14.952ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22672025-01-08T18:27:14.952ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
22682025-01-08T18:27:14.952ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 335 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57068 req_id = 4c5563c9-e886-49ee-adae-8a93e63ad007 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
22692025-01-08T18:27:25.660ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7761 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = 9ac51087-084c-4bf9-911c-45c6b8003ce6 response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
22702025-01-08T18:27:32.112ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:45690
22712025-01-08T18:27:32.113ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:45690 req_id = f91e10d8-d4d7-4894-b73b-3cf6834850b4 response_code = 200 uri = /baseboard
22722025-01-08T18:27:43.974ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42169
22732025-01-08T18:27:43.974ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42169 req_id = 2b000bea-4f24-4deb-8018-c89935482a02 response_code = 200 uri = /vpc-routes
22742025-01-08T18:27:43.992ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41994
22752025-01-08T18:27:43.992ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41994 req_id = 90e73472-5858-45a8-b5c0-106ea7fa6a3e response_code = 200 uri = /vpc-routes
22762025-01-08T18:27:44.028ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
22772025-01-08T18:27:44.028ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 80 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42169 req_id = 05cd9c1c-dfa1-4231-bd56-922f26d2124f response_code = 204 uri = /eip-gateways
22782025-01-08T18:27:44.038ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 306 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42169 req_id = 2c841988-ba81-4deb-9872-93c957b7c3e7 response_code = 204 uri = /vpc-routes
22792025-01-08T18:27:44.060ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
22802025-01-08T18:27:44.060ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 66 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41994 req_id = c67af837-7c75-4e8d-8239-23c9410b8645 response_code = 204 uri = /eip-gateways
22812025-01-08T18:27:44.070ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 294 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41994 req_id = 197eab80-8fd4-4e81-b5c8-2f046434e77f response_code = 204 uri = /vpc-routes
22822025-01-08T18:27:44.145ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61150
22832025-01-08T18:27:44.145ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61150 req_id = b92b4bd2-d1bd-4e89-985b-4d8cb929f129 response_code = 200 uri = /v2p
22842025-01-08T18:27:44.162ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58549
22852025-01-08T18:27:44.162ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58549 req_id = cbedaaa7-b14b-4d30-9e17-172abfb65af9 response_code = 200 uri = /v2p
22862025-01-08T18:27:44.163ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46019
22872025-01-08T18:27:44.163ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
22882025-01-08T18:27:44.200ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
22892025-01-08T18:27:44.256ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93079 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:46019 req_id = cd6a66c6-7385-4cd7-aa73-d357fa7f9210 response_code = 204 uri = /switch-ports
22902025-01-08T18:27:44.256ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 201 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46019 req_id = f845a4d5-535a-44ba-a1fa-3b1c5c1f8da6 response_code = 200 uri = /network-bootstore-config
22912025-01-08T18:27:44.266ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61590
22922025-01-08T18:27:44.266ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
22932025-01-08T18:27:44.316ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
22942025-01-08T18:27:44.395ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 129082 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:61590 req_id = 26219f70-0c36-4d11-b5bb-81437347124e response_code = 204 uri = /switch-ports
22952025-01-08T18:27:44.395ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 258 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61590 req_id = 3012cc7d-7fb9-443d-b109-66a89d0b0014 response_code = 200 uri = /network-bootstore-config
22962025-01-08T18:27:44.475ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47756
22972025-01-08T18:27:44.475ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47756 req_id = 2e906b98-3e70-4517-919d-9d93a8c2e4cd response_code = 200 uri = /vpc-routes
22982025-01-08T18:27:44.523ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
22992025-01-08T18:27:44.523ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47756 req_id = 27693277-e1bc-4ad1-b7fb-6edf336fafd6 response_code = 204 uri = /eip-gateways
23002025-01-08T18:27:44.526ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40332
23012025-01-08T18:27:44.526ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 164 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40332 req_id = 8676ed0a-ee25-43e3-9cc8-944a404a7b8e response_code = 200 uri = /v2p
23022025-01-08T18:27:44.532ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 411 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47756 req_id = 4cbac685-6da5-41b0-aeb1-6d544f0f6353 response_code = 204 uri = /vpc-routes
23032025-01-08T18:27:44.559ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53521
23042025-01-08T18:27:44.559ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
23052025-01-08T18:27:44.597ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
23062025-01-08T18:27:44.650ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90609 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:53521 req_id = e44bd0a6-e814-423a-a919-9c6bfe1703ab response_code = 204 uri = /switch-ports
23072025-01-08T18:27:44.650ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 195 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53521 req_id = 3929b0ed-8470-4bbc-95be-7d4a5401b318 response_code = 200 uri = /network-bootstore-config
23082025-01-08T18:27:55.660ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7703 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = d9e1710d-8c9f-4792-8b1b-e78f85944011 response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
23092025-01-08T18:28:02.147ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:40777
23102025-01-08T18:28:02.147ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:40777 req_id = d2d55782-f562-41a1-bd31-2491e8f71619 response_code = 200 uri = /baseboard
23112025-01-08T18:28:14.083ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44221
23122025-01-08T18:28:14.083ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 54 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44221 req_id = 52b5c42e-58a6-4aa7-bd0d-33c5caa6412e response_code = 200 uri = /vpc-routes
23132025-01-08T18:28:14.156ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
23142025-01-08T18:28:14.156ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44221 req_id = 9ceecdff-c07f-43b3-8193-a4a25159dd70 response_code = 204 uri = /eip-gateways
23152025-01-08T18:28:14.173ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38404
23162025-01-08T18:28:14.174ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38404 req_id = 11def8c4-10af-4c23-9003-cabb122151a6 response_code = 200 uri = /vpc-routes
23172025-01-08T18:28:14.221ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
23182025-01-08T18:28:14.221ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38404 req_id = 2ff4f9ca-ea25-4299-9f63-b4df062a5dc6 response_code = 204 uri = /eip-gateways
23192025-01-08T18:28:14.230ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 172 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38404 req_id = 7bfa4196-6cf1-42ad-a934-980f04239363 response_code = 204 uri = /vpc-routes
23202025-01-08T18:28:14.277ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55303
23212025-01-08T18:28:14.278ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55303 req_id = 01396f98-0f19-4d74-b32c-ab956efde415 response_code = 200 uri = /v2p
23222025-01-08T18:28:14.286ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63368
23232025-01-08T18:28:14.286ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63368 req_id = 1ce652d7-6fe0-4f6d-a5a4-8e8826539fd0 response_code = 200 uri = /v2p
23242025-01-08T18:28:14.300ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 227 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44221 req_id = bacfc3d8-74ae-41fc-92b7-d50ccdac0b31 response_code = 204 uri = /vpc-routes
23252025-01-08T18:28:14.382ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45695
23262025-01-08T18:28:14.382ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
23272025-01-08T18:28:14.424ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
23282025-01-08T18:28:14.479ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96952 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:45695 req_id = 5173a582-16ee-4ac4-a81f-773f0edf061a response_code = 204 uri = /switch-ports
23292025-01-08T18:28:14.479ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50984
23302025-01-08T18:28:14.479ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
23312025-01-08T18:28:14.482ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3316 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45695 req_id = e5ef3859-a1ea-4cdc-8d0c-17cdb61db939 response_code = 200 uri = /network-bootstore-config
23322025-01-08T18:28:14.515ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
23332025-01-08T18:28:14.565ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86568 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:50984 req_id = d27b3edb-8bcc-4402-9bb5-1bb26b424bef response_code = 204 uri = /switch-ports
23342025-01-08T18:28:14.566ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50984 req_id = e73567e1-7495-4b37-9002-6007e2232b38 response_code = 200 uri = /network-bootstore-config
23352025-01-08T18:28:14.600ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42163
23362025-01-08T18:28:14.601ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 54 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42163 req_id = 6d033fbd-e887-459b-8246-e7e62fb39f8d response_code = 200 uri = /vpc-routes
23372025-01-08T18:28:14.663ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
23382025-01-08T18:28:14.664ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 77 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42163 req_id = 996630f9-2905-4b64-8a3a-76b131f83588 response_code = 204 uri = /eip-gateways
23392025-01-08T18:28:14.676ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 199 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42163 req_id = 0290efda-c6da-417c-ba74-4820a8ea40c9 response_code = 204 uri = /vpc-routes
23402025-01-08T18:28:14.800ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33142
23412025-01-08T18:28:14.800ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
23422025-01-08T18:28:14.837ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
23432025-01-08T18:28:14.888ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87499 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:33142 req_id = 88a0d196-7b3f-48ae-bf23-2a90fadfdfea response_code = 204 uri = /switch-ports
23442025-01-08T18:28:14.888ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51004
23452025-01-08T18:28:14.888ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51004 req_id = f3d0bc63-3339-47c3-addf-2992a37b3c1b response_code = 200 uri = /v2p
23462025-01-08T18:28:14.888ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33142 req_id = be295c8a-3189-4ae0-a236-9b96a5cadb6e response_code = 200 uri = /network-bootstore-config
23472025-01-08T18:28:25.661ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8113 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = f6b4182b-f7a3-4e85-b641-9f0be0981b0b response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
23482025-01-08T18:28:32.180ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:39626
23492025-01-08T18:28:32.180ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:39626 req_id = 42bfc5d2-2a34-4844-b52c-a4bbd18d3ec7 response_code = 200 uri = /baseboard
23502025-01-08T18:28:44.037ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44212
23512025-01-08T18:28:44.037ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44212 req_id = 28a36af8-f90a-413d-a3f2-c1bdfea4db3f response_code = 200 uri = /vpc-routes
23522025-01-08T18:28:44.044ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56772
23532025-01-08T18:28:44.044ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56772 req_id = 40664486-9f0e-43e1-a61e-b29c003ba561 response_code = 200 uri = /vpc-routes
23542025-01-08T18:28:44.070ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58621
23552025-01-08T18:28:44.070ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 138 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58621 req_id = 4383342e-e79e-46fc-97d2-bf236f24807d response_code = 200 uri = /v2p
23562025-01-08T18:28:44.080ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
23572025-01-08T18:28:44.080ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44212 req_id = f6a11d96-60f3-4916-b536-57b8106dd59c response_code = 204 uri = /eip-gateways
23582025-01-08T18:28:44.086ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
23592025-01-08T18:28:44.086ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 43 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56772 req_id = 284a2050-2def-407e-a28e-96de06d6a991 response_code = 204 uri = /eip-gateways
23602025-01-08T18:28:44.090ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 181 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44212 req_id = 88416222-96eb-49c8-bf8c-75b03ae4c84f response_code = 204 uri = /vpc-routes
23612025-01-08T18:28:44.095ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56772 req_id = 80affd00-d64f-4464-8105-edeb64ef025b response_code = 204 uri = /vpc-routes
23622025-01-08T18:28:44.117ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40061
23632025-01-08T18:28:44.117ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
23642025-01-08T18:28:44.157ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
23652025-01-08T18:28:44.213ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96200 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:40061 req_id = 51c3bfbf-72d3-4391-a248-fc2bd63da7cf response_code = 204 uri = /switch-ports
23662025-01-08T18:28:44.213ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56641
23672025-01-08T18:28:44.213ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57048
23682025-01-08T18:28:44.214ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 234 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57048 req_id = 21c5c857-c008-4ff0-a1e8-c5b43af85f1d response_code = 200 uri = /v2p
23692025-01-08T18:28:44.214ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 312 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40061 req_id = 9b8350e7-2a6a-45ab-afb5-5be27246b37a response_code = 200 uri = /network-bootstore-config
23702025-01-08T18:28:44.214ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
23712025-01-08T18:28:44.252ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
23722025-01-08T18:28:44.330ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 115888 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:56641 req_id = 6664e89e-9c5c-486e-bab9-3af520126eac response_code = 204 uri = /switch-ports
23732025-01-08T18:28:44.330ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56641 req_id = 9a0e908b-01a6-4995-b643-e84c028738c0 response_code = 200 uri = /network-bootstore-config
23742025-01-08T18:28:44.502ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48224
23752025-01-08T18:28:44.502ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48224 req_id = a3f51b5d-2b40-4496-9dfc-67be5f72de66 response_code = 200 uri = /vpc-routes
23762025-01-08T18:28:44.554ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
23772025-01-08T18:28:44.554ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48224 req_id = fcb34879-ed9d-4982-893b-64ff539d0d1e response_code = 204 uri = /eip-gateways
23782025-01-08T18:28:44.565ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 260 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48224 req_id = baa148a8-b549-4e02-9f96-9803daaaa4cf response_code = 204 uri = /vpc-routes
23792025-01-08T18:28:44.619ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59624
23802025-01-08T18:28:44.619ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
23812025-01-08T18:28:44.656ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
23822025-01-08T18:28:44.710ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91598 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:59624 req_id = 719d8816-ab42-4383-b139-3bb7d75c646b response_code = 204 uri = /switch-ports
23832025-01-08T18:28:44.711ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34025
23842025-01-08T18:28:44.711ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 220 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59624 req_id = 6bc9dda9-1f89-43d0-bbef-4084b1e26c7e response_code = 200 uri = /network-bootstore-config
23852025-01-08T18:28:44.711ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 222 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34025 req_id = 84897e77-315d-46dd-98ae-025c7f82d88b response_code = 200 uri = /v2p
23862025-01-08T18:28:55.661ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8199 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = 9db4b8b7-39a6-4b21-bda6-a811f468e09d response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
23872025-01-08T18:29:02.214ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:53219
23882025-01-08T18:29:02.214ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:53219 req_id = 7a9c0db6-a48a-4fa9-bcf9-85f91c8e2806 response_code = 200 uri = /baseboard
23892025-01-08T18:29:14.128ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63454
23902025-01-08T18:29:14.128ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 66 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63454 req_id = b7f24665-8924-4cbb-b90b-b27ab0fb2a86 response_code = 200 uri = /vpc-routes
23912025-01-08T18:29:14.181ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
23922025-01-08T18:29:14.181ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 80 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63454 req_id = c32ed221-0b92-4a59-af72-396e0e674ad1 response_code = 204 uri = /eip-gateways
23932025-01-08T18:29:14.191ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 201 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63454 req_id = d0803e55-5704-4d5d-8817-96a44ae0a136 response_code = 204 uri = /vpc-routes
23942025-01-08T18:29:14.312ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41376
23952025-01-08T18:29:14.312ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41376 req_id = a41fc090-a799-4bcd-a29e-629c7db549ce response_code = 200 uri = /vpc-routes
23962025-01-08T18:29:14.385ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
23972025-01-08T18:29:14.386ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 189 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41376 req_id = ac59b628-9284-41d1-af13-ba07642cc58d response_code = 204 uri = /eip-gateways
23982025-01-08T18:29:14.394ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45423
23992025-01-08T18:29:14.394ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
24002025-01-08T18:29:14.445ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
24012025-01-08T18:29:14.512ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 118016 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:45423 req_id = 4d32fe58-48d1-4931-877c-63d93f8425a1 response_code = 204 uri = /switch-ports
24022025-01-08T18:29:14.512ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57610
24032025-01-08T18:29:14.512ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57517
24042025-01-08T18:29:14.512ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 310 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45423 req_id = 8b6a574a-80a8-408a-a096-994ff8687faf response_code = 200 uri = /network-bootstore-config
24052025-01-08T18:29:14.513ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 299 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57610 req_id = 748ca91f-ef68-4b77-b114-506903832781 response_code = 200 uri = /v2p
24062025-01-08T18:29:14.513ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 271 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57517 req_id = e40c71b8-0dd9-4e2d-a67a-dc453134666a response_code = 200 uri = /v2p
24072025-01-08T18:29:14.513ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 555 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41376 req_id = 9b535a64-3dde-42d0-9e1d-8574499f6410 response_code = 204 uri = /vpc-routes
24082025-01-08T18:29:14.577ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44465
24092025-01-08T18:29:14.577ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
24102025-01-08T18:29:14.611ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
24112025-01-08T18:29:14.663ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 85953 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:44465 req_id = 12992fa3-41dd-4741-9b1b-80b6bf158e6a response_code = 204 uri = /switch-ports
24122025-01-08T18:29:14.663ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59919
24132025-01-08T18:29:14.663ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59919 req_id = fb984a71-9151-4f46-8203-b22878ee6f29 response_code = 200 uri = /vpc-routes
24142025-01-08T18:29:14.663ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44465 req_id = a81db290-35d4-4f14-adbe-a6f79ed3422d response_code = 200 uri = /network-bootstore-config
24152025-01-08T18:29:14.682ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
24162025-01-08T18:29:14.683ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59919 req_id = f3fcdac8-fa6a-4f94-ae7f-c105144181fd response_code = 204 uri = /eip-gateways
24172025-01-08T18:29:14.692ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 320 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59919 req_id = 6e7605fe-00f7-4678-97cd-845c87480e1c response_code = 204 uri = /vpc-routes
24182025-01-08T18:29:14.827ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52801
24192025-01-08T18:29:14.827ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52801 req_id = b853b0a5-3865-4e48-b14e-d44cfafbb9ef response_code = 200 uri = /v2p
24202025-01-08T18:29:14.839ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53070
24212025-01-08T18:29:14.840ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
24222025-01-08T18:29:14.872ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
24232025-01-08T18:29:14.920ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 80642 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:53070 req_id = 63d04cf8-e245-4c09-a72a-d9257a4315b2 response_code = 204 uri = /switch-ports
24242025-01-08T18:29:14.920ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53070 req_id = be479ee4-195c-4f0c-a2fd-513c668a60fa response_code = 200 uri = /network-bootstore-config
24252025-01-08T18:29:25.661ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7553 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = 5cb84208-7dc7-4ef2-b63b-d76ae29a5ec9 response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
24262025-01-08T18:29:32.245ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58839
24272025-01-08T18:29:32.245ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 74 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58839 req_id = a157caff-76d3-44ff-bdb6-14b0d280eebf response_code = 200 uri = /baseboard
24282025-01-08T18:29:44.045ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36801
24292025-01-08T18:29:44.045ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36801 req_id = 849bed8f-4b20-4da5-968b-870e78b2afaf response_code = 200 uri = /vpc-routes
24302025-01-08T18:29:44.090ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
24312025-01-08T18:29:44.090ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36801 req_id = aa083bbe-b8db-4f26-b870-ea19abe875e7 response_code = 204 uri = /eip-gateways
24322025-01-08T18:29:44.099ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36801 req_id = 589866f9-2b85-4bcb-ae05-38a04b4a2e8a response_code = 204 uri = /vpc-routes
24332025-01-08T18:29:44.101ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47772
24342025-01-08T18:29:44.101ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
24352025-01-08T18:29:44.144ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
24362025-01-08T18:29:44.204ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103101 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:47772 req_id = 963f68ac-f22b-42fe-b220-22ca152db209 response_code = 204 uri = /switch-ports
24372025-01-08T18:29:44.204ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58950
24382025-01-08T18:29:44.204ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63323
24392025-01-08T18:29:44.204ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 211 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63323 req_id = 38515461-8760-4b3a-9d89-9ac4be0eaa73 response_code = 200 uri = /v2p
24402025-01-08T18:29:44.204ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 165 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58950 req_id = 32c6575a-b04f-4604-ad00-c382aa33e121 response_code = 200 uri = /vpc-routes
24412025-01-08T18:29:44.204ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 277 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47772 req_id = f85272c0-b8e5-4dd8-8a4a-d60d951c7ce9 response_code = 200 uri = /network-bootstore-config
24422025-01-08T18:29:44.245ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
24432025-01-08T18:29:44.245ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:58950 req_id = 6616d425-19ff-4278-988d-6d896a5e7cba response_code = 204 uri = /eip-gateways
24442025-01-08T18:29:44.265ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55434
24452025-01-08T18:29:44.265ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55434 req_id = e29cd372-d78a-479b-8f37-aeaba899a2b7 response_code = 200 uri = /v2p
24462025-01-08T18:29:44.271ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 179 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:58950 req_id = 11fae09a-22d6-4d82-90df-be6a18643c9a response_code = 204 uri = /vpc-routes
24472025-01-08T18:29:44.281ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64076
24482025-01-08T18:29:44.281ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
24492025-01-08T18:29:44.335ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
24502025-01-08T18:29:44.404ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 123123 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:64076 req_id = f4432823-a472-47b0-864a-08ccbddef15c response_code = 204 uri = /switch-ports
24512025-01-08T18:29:44.405ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 152 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64076 req_id = cb79aa2b-6198-45b5-8d52-eef196d110f8 response_code = 200 uri = /network-bootstore-config
24522025-01-08T18:29:44.466ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47877
24532025-01-08T18:29:44.467ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47877 req_id = 5cd6bca1-6dc9-45b6-8385-ee2cb54d18fb response_code = 200 uri = /vpc-routes
24542025-01-08T18:29:44.528ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50514
24552025-01-08T18:29:44.528ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50514 req_id = 5f46147d-f6b6-4bcf-b417-e0ac0e0a7096 response_code = 200 uri = /v2p
24562025-01-08T18:29:44.528ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
24572025-01-08T18:29:44.528ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47877 req_id = cba9f37d-a4c6-44ac-a151-52b4ea55af02 response_code = 204 uri = /eip-gateways
24582025-01-08T18:29:44.536ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 226 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47877 req_id = 21a3c089-3224-4158-94f1-95baa2b71f40 response_code = 204 uri = /vpc-routes
24592025-01-08T18:29:44.565ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56938
24602025-01-08T18:29:44.565ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
24612025-01-08T18:29:44.605ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
24622025-01-08T18:29:44.662ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97003 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:56938 req_id = 97538ebf-aa52-4866-8647-b4b3144a77d3 response_code = 204 uri = /switch-ports
24632025-01-08T18:29:44.662ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56938 req_id = cacc29a5-76db-4062-a06e-454bc525f83b response_code = 200 uri = /network-bootstore-config
24642025-01-08T18:29:55.661ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7304 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = d9d7eea0-f229-44d4-b949-7f11236350e2 response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
24652025-01-08T18:29:55.941ZINFOSledAgent (auto-cleanup-task): running automatic periodic zone bundle cleanup file = sled-agent/src/zone_bundle.rs:197
24662025-01-08T18:29:55.941ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
24672025-01-08T18:29:55.972ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 4769, tv_nsec: 963522963 } time_to_next_cleanup = 599.99999666s
24682025-01-08T18:30:02.290ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:47274
24692025-01-08T18:30:02.290ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 55 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:47274 req_id = 8cbb345c-2a9d-4dee-8bc6-af952c8eed3a response_code = 200 uri = /baseboard
24702025-01-08T18:30:10.339ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
24712025-01-08T18:30:14.209ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55602
24722025-01-08T18:30:14.209ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 73 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55602 req_id = 084d99d7-fd8b-4916-af47-58c1775e5fea response_code = 200 uri = /vpc-routes
24732025-01-08T18:30:14.215ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34340
24742025-01-08T18:30:14.215ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34340 req_id = 26e32d8e-ea6c-47ae-b75b-14ba4624004f response_code = 200 uri = /vpc-routes
24752025-01-08T18:30:14.226ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
24762025-01-08T18:30:14.226ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55602 req_id = 0587bf68-0180-474d-99dc-b93ccf954fcf response_code = 204 uri = /eip-gateways
24772025-01-08T18:30:14.237ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 211 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55602 req_id = 0a7fa17f-42f1-40e2-aa4e-ca28c340f045 response_code = 204 uri = /vpc-routes
24782025-01-08T18:30:14.269ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
24792025-01-08T18:30:14.269ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34340 req_id = f7a985f6-5002-4d37-b981-77ef86c63b36 response_code = 204 uri = /eip-gateways
24802025-01-08T18:30:14.281ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 254 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34340 req_id = a0e7d9f7-df90-4ed2-8e8d-785595cc0726 response_code = 204 uri = /vpc-routes
24812025-01-08T18:30:14.348ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56137
24822025-01-08T18:30:14.348ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 161 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56137 req_id = 9bce805b-2747-402f-a070-dcef6770b670 response_code = 200 uri = /v2p
24832025-01-08T18:30:14.463ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57510
24842025-01-08T18:30:14.463ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57510 req_id = e7cd9c0c-fb93-4b52-9d92-d81dfac2b50e response_code = 200 uri = /v2p
24852025-01-08T18:30:14.479ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52144
24862025-01-08T18:30:14.479ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
24872025-01-08T18:30:14.518ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
24882025-01-08T18:30:14.575ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96000 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:52144 req_id = 6f2d9c4c-981a-40cb-9652-7e29bdc4329b response_code = 204 uri = /switch-ports
24892025-01-08T18:30:14.575ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54240
24902025-01-08T18:30:14.575ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52144 req_id = d9adf14f-cd9a-4468-87b2-f26cd90f07d7 response_code = 200 uri = /network-bootstore-config
24912025-01-08T18:30:14.576ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
24922025-01-08T18:30:14.612ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
24932025-01-08T18:30:14.668ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92931 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:54240 req_id = 120db270-2477-455a-9091-9ad13c5b4ad0 response_code = 204 uri = /switch-ports
24942025-01-08T18:30:14.669ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 184 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54240 req_id = 1ffcd9a8-6c66-4eea-bfd6-26c79de8e8af response_code = 200 uri = /network-bootstore-config
24952025-01-08T18:30:14.683ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61931
24962025-01-08T18:30:14.683ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 54 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61931 req_id = 2cd44d6c-faf2-49f8-be7b-ce78aa8d612a response_code = 200 uri = /vpc-routes
24972025-01-08T18:30:14.719ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60375
24982025-01-08T18:30:14.719ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60375 req_id = 809f0858-a6e0-473b-a3c0-07b637e035d1 response_code = 200 uri = /v2p
24992025-01-08T18:30:14.733ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
25002025-01-08T18:30:14.733ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61931 req_id = 85b62b46-52ad-4af1-94f0-6550101b1dc9 response_code = 204 uri = /eip-gateways
25012025-01-08T18:30:14.742ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 174 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61931 req_id = 3b90fa67-2f23-4a54-ab1a-81e531829692 response_code = 204 uri = /vpc-routes
25022025-01-08T18:30:14.821ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36011
25032025-01-08T18:30:14.821ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
25042025-01-08T18:30:14.855ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
25052025-01-08T18:30:14.906ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 85158 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:36011 req_id = f2397d38-275a-4bff-90ef-69d5dc5923dc response_code = 204 uri = /switch-ports
25062025-01-08T18:30:14.906ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 122 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36011 req_id = 3b6dd086-304f-4f5e-92f7-22edfcdbe229 response_code = 200 uri = /network-bootstore-config
25072025-01-08T18:30:25.661ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6642 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = ba6557d1-f0a1-4b97-a465-f019673193af response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
25082025-01-08T18:30:32.325ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:44138
25092025-01-08T18:30:32.325ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 50 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:44138 req_id = 1844cd5e-cd70-4b38-81df-2eb1e4754daa response_code = 200 uri = /baseboard
25102025-01-08T18:30:44.048ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41127
25112025-01-08T18:30:44.048ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41127 req_id = 945ec648-c663-4888-b543-dfd4b70d1127 response_code = 200 uri = /vpc-routes
25122025-01-08T18:30:44.050ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46375
25132025-01-08T18:30:44.051ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 27 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46375 req_id = 4dc89048-5cda-4f0c-a664-f8f4aeca8183 response_code = 200 uri = /vpc-routes
25142025-01-08T18:30:44.065ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
25152025-01-08T18:30:44.065ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46375 req_id = 9e2630a0-a41f-47e7-8fcf-077fe5ce2cb4 response_code = 204 uri = /eip-gateways
25162025-01-08T18:30:44.080ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 261 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46375 req_id = b3245d18-4bb9-4f0f-991b-3f65b1d0d22d response_code = 204 uri = /vpc-routes
25172025-01-08T18:30:44.094ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
25182025-01-08T18:30:44.094ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41127 req_id = 722cbbd3-8150-4a69-b165-6fd3910219de response_code = 204 uri = /eip-gateways
25192025-01-08T18:30:44.104ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 194 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41127 req_id = 312b1d95-88aa-4046-a605-75a659b3b910 response_code = 204 uri = /vpc-routes
25202025-01-08T18:30:44.155ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62416
25212025-01-08T18:30:44.155ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62416 req_id = c0fe826b-4a33-401a-86ac-39ba1629b1a2 response_code = 200 uri = /v2p
25222025-01-08T18:30:44.162ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63652
25232025-01-08T18:30:44.162ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
25242025-01-08T18:30:44.201ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
25252025-01-08T18:30:44.257ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94996 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:63652 req_id = c38e1781-090e-4817-baa0-dc037696cb69 response_code = 204 uri = /switch-ports
25262025-01-08T18:30:44.257ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54024
25272025-01-08T18:30:44.257ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44615
25282025-01-08T18:30:44.257ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
25292025-01-08T18:30:44.261ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3161 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54024 req_id = 8cc56bca-356b-4a2f-8f0a-ab4d3ec614b3 response_code = 200 uri = /v2p
25302025-01-08T18:30:44.261ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3293 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63652 req_id = 06a0f764-efe9-4bd8-9449-0b03d5c1303d response_code = 200 uri = /network-bootstore-config
25312025-01-08T18:30:44.299ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
25322025-01-08T18:30:44.362ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104940 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:44615 req_id = 0c8b66d5-ea4c-4ec9-97a1-bb77a7e337d5 response_code = 204 uri = /switch-ports
25332025-01-08T18:30:44.363ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 179 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44615 req_id = e18a1675-3251-43c8-9578-344cccc23abb response_code = 200 uri = /network-bootstore-config
25342025-01-08T18:30:44.548ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60803
25352025-01-08T18:30:44.548ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60803 req_id = 1b184126-ff2d-4fa8-97aa-bd19451a505c response_code = 200 uri = /vpc-routes
25362025-01-08T18:30:44.578ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53576
25372025-01-08T18:30:44.578ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53576 req_id = 77ff36cb-3729-423f-8e31-19320ab4dc0d response_code = 200 uri = /v2p
25382025-01-08T18:30:44.631ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
25392025-01-08T18:30:44.631ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 81 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:60803 req_id = 0751f950-979f-475f-a704-53c0d5a1b53d response_code = 204 uri = /eip-gateways
25402025-01-08T18:30:44.640ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 199 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:60803 req_id = c8ec3b23-024d-4f2b-94ff-03da446814f5 response_code = 204 uri = /vpc-routes
25412025-01-08T18:30:44.735ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60456
25422025-01-08T18:30:44.735ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
25432025-01-08T18:30:44.773ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
25442025-01-08T18:30:44.824ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89506 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:60456 req_id = 0cffc604-4f59-49d1-8f10-1581d8e11933 response_code = 204 uri = /switch-ports
25452025-01-08T18:30:44.825ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 232 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60456 req_id = c59c1fd6-9367-4313-b6fa-ae524d45d195 response_code = 200 uri = /network-bootstore-config
25462025-01-08T18:30:55.662ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7581 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = 798f7c12-7fac-4259-b582-4fc0e2a905e4 response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
25472025-01-08T18:31:02.359ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:41509
25482025-01-08T18:31:02.359ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:41509 req_id = 17c481e2-d63f-4a16-b462-7a65ed9b5b22 response_code = 200 uri = /baseboard
25492025-01-08T18:31:14.084ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48314
25502025-01-08T18:31:14.084ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48314 req_id = b552914e-805b-4500-a520-5812f0e85758 response_code = 200 uri = /vpc-routes
25512025-01-08T18:31:14.100ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
25522025-01-08T18:31:14.100ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 73 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48314 req_id = 292e78e4-ea44-44f8-852e-a9a4d7400fc8 response_code = 204 uri = /eip-gateways
25532025-01-08T18:31:14.113ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 285 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48314 req_id = 6aca3106-20cf-4726-9fc7-fec830ddbd4c response_code = 204 uri = /vpc-routes
25542025-01-08T18:31:14.176ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33315
25552025-01-08T18:31:14.176ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 39 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33315 req_id = 1b22c73c-3106-4d88-83ea-5d7b512f2409 response_code = 200 uri = /vpc-routes
25562025-01-08T18:31:14.190ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
25572025-01-08T18:31:14.190ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:33315 req_id = d91c6828-e103-47f0-8fb2-7b5268b3352b response_code = 204 uri = /eip-gateways
25582025-01-08T18:31:14.274ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61268
25592025-01-08T18:31:14.274ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 120 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61268 req_id = 57c31d31-a7f4-4ab6-8de7-85051fb87c9f response_code = 200 uri = /v2p
25602025-01-08T18:31:14.277ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 183 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:33315 req_id = de57c448-ca9a-409a-80e9-f24a1dab4553 response_code = 204 uri = /vpc-routes
25612025-01-08T18:31:14.287ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34741
25622025-01-08T18:31:14.287ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34741 req_id = f0c08779-ddd1-486f-96d3-be664112304f response_code = 200 uri = /v2p
25632025-01-08T18:31:14.400ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34499
25642025-01-08T18:31:14.400ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
25652025-01-08T18:31:14.445ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
25662025-01-08T18:31:14.503ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103028 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:34499 req_id = 0d3bd353-aaf0-48e6-bb37-3be29cce6680 response_code = 204 uri = /switch-ports
25672025-01-08T18:31:14.503ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58795
25682025-01-08T18:31:14.503ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34499 req_id = 0fdff9f4-3b63-4757-b33a-d84fad99a6df response_code = 200 uri = /network-bootstore-config
25692025-01-08T18:31:14.503ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
25702025-01-08T18:31:14.542ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
25712025-01-08T18:31:14.599ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95590 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:58795 req_id = 0c95b5d7-e4bd-47ee-bf57-865091a66186 response_code = 204 uri = /switch-ports
25722025-01-08T18:31:14.599ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 151 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58795 req_id = 33c9995c-ba7e-4fc0-9de6-4cc14911e1af response_code = 200 uri = /network-bootstore-config
25732025-01-08T18:31:14.699ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64354
25742025-01-08T18:31:14.699ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64354 req_id = 47c7da0d-10ea-4a12-8916-6c0078ca2688 response_code = 200 uri = /vpc-routes
25752025-01-08T18:31:14.714ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
25762025-01-08T18:31:14.714ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64354 req_id = 10c38b04-8c17-4c2d-8be4-2ae17cbfd2a2 response_code = 204 uri = /eip-gateways
25772025-01-08T18:31:14.739ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40985
25782025-01-08T18:31:14.739ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40985 req_id = 9787d696-768c-4982-b59c-c0f84f849a59 response_code = 200 uri = /v2p
25792025-01-08T18:31:14.748ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 225 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:64354 req_id = 1f996f9a-832e-4c3b-9968-bacfc74290ca response_code = 204 uri = /vpc-routes
25802025-01-08T18:31:14.820ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39566
25812025-01-08T18:31:14.820ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
25822025-01-08T18:31:14.857ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
25832025-01-08T18:31:14.910ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89930 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:39566 req_id = 05453ceb-8ab8-4862-be5d-8d8ea975bd32 response_code = 204 uri = /switch-ports
25842025-01-08T18:31:14.911ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 215 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39566 req_id = 833d7053-196e-4160-aeb5-1482c3115905 response_code = 200 uri = /network-bootstore-config
25852025-01-08T18:31:25.662ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7971 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = 769c8967-754b-4b45-827f-81c635cea87a response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
25862025-01-08T18:31:32.392ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58242
25872025-01-08T18:31:32.392ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 55 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:58242 req_id = 31afe0c1-d52a-4a22-a360-634a490899ed response_code = 200 uri = /baseboard
25882025-01-08T18:31:44.077ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37967
25892025-01-08T18:31:44.077ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37967 req_id = 3b2bf9f5-55b9-4919-86a4-812ac64941c9 response_code = 200 uri = /vpc-routes
25902025-01-08T18:31:44.092ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60043
25912025-01-08T18:31:44.092ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60043 req_id = d76b9328-5245-4767-96d7-6fc2a85bad54 response_code = 200 uri = /vpc-routes
25922025-01-08T18:31:44.098ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
25932025-01-08T18:31:44.098ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37967 req_id = 35ec0d64-6ffc-4dda-99ce-23650345c006 response_code = 204 uri = /eip-gateways
25942025-01-08T18:31:44.112ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 229 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37967 req_id = f551159c-6faa-4de8-ad7c-e684f3c4c16e response_code = 204 uri = /vpc-routes
25952025-01-08T18:31:44.118ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
25962025-01-08T18:31:44.118ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:60043 req_id = eddac743-fcc0-44dc-a422-067b03b30377 response_code = 204 uri = /eip-gateways
25972025-01-08T18:31:44.135ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 169 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:60043 req_id = e3b9e3cc-4022-4d7b-8a70-8bbd9fb9137a response_code = 204 uri = /vpc-routes
25982025-01-08T18:31:44.204ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64083
25992025-01-08T18:31:44.204ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
26002025-01-08T18:31:44.245ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
26012025-01-08T18:31:44.312ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108177 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:64083 req_id = d58fefe7-d05a-4c3c-94f7-a65dab9b665c response_code = 204 uri = /switch-ports
26022025-01-08T18:31:44.312ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59959
26032025-01-08T18:31:44.312ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33710
26042025-01-08T18:31:44.312ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64918
26052025-01-08T18:31:44.313ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 222 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64918 req_id = 4d8eb27c-b2e1-4949-8433-f2572d43c8f6 response_code = 200 uri = /v2p
26062025-01-08T18:31:44.313ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
26072025-01-08T18:31:44.313ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 357 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64083 req_id = fda9eac0-5215-42e4-ba5c-cf9a7135b2bf response_code = 200 uri = /network-bootstore-config
26082025-01-08T18:31:44.313ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 267 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59959 req_id = caf24ed2-f9b3-4782-9f28-ab78a87819ba response_code = 200 uri = /v2p
26092025-01-08T18:31:44.362ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
26102025-01-08T18:31:44.427ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113859 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:33710 req_id = 061b7ab0-0f1f-4c74-b1fd-1b26b4bc05f5 response_code = 204 uri = /switch-ports
26112025-01-08T18:31:44.427ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33710 req_id = 02543a3d-d397-41c4-831e-acc81a60fcf3 response_code = 200 uri = /network-bootstore-config
26122025-01-08T18:31:44.478ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36165
26132025-01-08T18:31:44.478ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36165 req_id = 2ea761cb-20ef-4d2d-8bcf-e305d5062f23 response_code = 200 uri = /vpc-routes
26142025-01-08T18:31:44.521ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
26152025-01-08T18:31:44.521ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 80 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:36165 req_id = 1bc005d3-059a-4937-98e5-9bbff879639a response_code = 204 uri = /eip-gateways
26162025-01-08T18:31:44.530ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 224 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:36165 req_id = f11e5376-3641-4ec3-baca-40800a91acf9 response_code = 204 uri = /vpc-routes
26172025-01-08T18:31:44.561ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42200
26182025-01-08T18:31:44.561ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
26192025-01-08T18:31:44.601ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
26202025-01-08T18:31:44.657ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96238 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:42200 req_id = 32e68f39-8669-4f87-9347-de1612be2275 response_code = 204 uri = /switch-ports
26212025-01-08T18:31:44.658ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42200 req_id = 7f893017-a488-43c0-bb7c-da11fe7a9bce response_code = 200 uri = /network-bootstore-config
26222025-01-08T18:31:44.667ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41329
26232025-01-08T18:31:44.667ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 157 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41329 req_id = a208ab62-ac74-4969-ab47-9936e111f592 response_code = 200 uri = /v2p
26242025-01-08T18:31:55.662ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7545 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = 485a4802-6f85-432c-bf7b-ffcfec5756b1 response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
26252025-01-08T18:32:02.425ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43794
26262025-01-08T18:32:02.425ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:43794 req_id = 58464457-fce9-453e-9987-589af636ad21 response_code = 200 uri = /baseboard
26272025-01-08T18:32:14.268ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49433
26282025-01-08T18:32:14.269ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53066
26292025-01-08T18:32:14.269ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53066 req_id = 48db3fda-4884-4213-9dd9-ff1fdfaf164e response_code = 200 uri = /vpc-routes
26302025-01-08T18:32:14.269ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49433 req_id = b22758a7-a37a-4821-bd34-e7301d55ba9a response_code = 200 uri = /vpc-routes
26312025-01-08T18:32:14.305ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
26322025-01-08T18:32:14.305ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 151 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53066 req_id = c1a22541-3060-43e0-ae51-aa1351d84762 response_code = 204 uri = /eip-gateways
26332025-01-08T18:32:14.308ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
26342025-01-08T18:32:14.308ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49433 req_id = bd97550c-f76a-4006-8060-59888504c39c response_code = 204 uri = /eip-gateways
26352025-01-08T18:32:14.327ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 253 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49433 req_id = 7f6c171f-6057-433e-9a8f-6ed7ad366e80 response_code = 204 uri = /vpc-routes
26362025-01-08T18:32:14.328ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 185 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53066 req_id = 69f47b41-2f6f-4677-b127-3d3a677c3af4 response_code = 204 uri = /vpc-routes
26372025-01-08T18:32:14.409ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56198
26382025-01-08T18:32:14.409ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
26392025-01-08T18:32:14.461ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
26402025-01-08T18:32:14.530ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 120678 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:56198 req_id = 04bfddbc-3c78-458b-a5e4-510f0da5077c response_code = 204 uri = /switch-ports
26412025-01-08T18:32:14.530ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63516
26422025-01-08T18:32:14.530ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52538
26432025-01-08T18:32:14.530ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60216
26442025-01-08T18:32:14.531ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 332 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56198 req_id = 8a2f3826-d6a1-48d6-955f-59ef75508422 response_code = 200 uri = /network-bootstore-config
26452025-01-08T18:32:14.531ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 239 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60216 req_id = 4ee0fcc0-2b03-4e3e-8fad-2942d9bb8fea response_code = 200 uri = /v2p
26462025-01-08T18:32:14.531ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 216 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52538 req_id = c78f8610-5c49-4c53-90a9-c607ec99f39f response_code = 200 uri = /v2p
26472025-01-08T18:32:14.531ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
26482025-01-08T18:32:14.531ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
26492025-01-08T18:32:14.532ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-zones.json file = common/src/ledger.rs:177
26502025-01-08T18:32:14.533ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-zones.json file = common/src/ledger.rs:177
26512025-01-08T18:32:14.547ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
26522025-01-08T18:32:14.547ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f
26532025-01-08T18:32:14.572ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49171
26542025-01-08T18:32:14.572ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
26552025-01-08T18:32:14.572ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
26562025-01-08T18:32:14.572ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-zones.json file = common/src/ledger.rs:177
26572025-01-08T18:32:14.572ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-zones.json file = common/src/ledger.rs:177
26582025-01-08T18:32:14.577ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
26592025-01-08T18:32:14.577ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_0ac8ff68-15ea-4477-a206-5a7619973e85
26602025-01-08T18:32:14.594ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
26612025-01-08T18:32:14.594ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f
26622025-01-08T18:32:14.614ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
26632025-01-08T18:32:14.614ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_bf665b71-2255-48d6-b1fd-c863b4c8e1af
26642025-01-08T18:32:14.627ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62285
26652025-01-08T18:32:14.627ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0041456a-da2f-4ef8-bdf7-8e18ca9d8279, kind: Service { id: 1c3a7513-ed19-483c-872f-3377ba80f37f }, name: Name("external-dns-1c3a7513-ed19-483c-872f-3377ba80f37f"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 154, 170])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ce1a1751-7375-4744-b6f0-690b02866c95, kind: Service { id: 3963284d-f267-448b-bfbb-a55402c7ef81 }, name: Name("external-dns-3963284d-f267-448b-bfbb-a55402c7ef81"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 116])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2beb6270-a9c4-41a0-b3ab-eeff6ddd3582, kind: Service { id: cef4a321-387e-44a1-92c9-23673073cdd8 }, name: Name("nexus-cef4a321-387e-44a1-92c9-23673073cdd8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 113])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bfb8fa95-f4f1-4ec3-bb77-0e8adbc6ac68, kind: Service { id: a28f8263-3fdd-4cd1-a79d-6ef15080a80e }, name: Name("nexus-a28f8263-3fdd-4cd1-a79d-6ef15080a80e"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c656ac3d-ffdc-434e-863c-8efcfbd82ab4, kind: Service { id: 099ecd74-9dbb-4a7f-94ce-79177e3bb646 }, name: Name("nexus-099ecd74-9dbb-4a7f-94ce-79177e3bb646"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 171])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
26662025-01-08T18:32:14.627ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
26672025-01-08T18:32:14.627ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
26682025-01-08T18:32:14.627ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
26692025-01-08T18:32:14.627ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
26702025-01-08T18:32:14.627ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
26712025-01-08T18:32:14.627ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
26722025-01-08T18:32:14.627ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 444 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:62285 req_id = 27e10bd8-21ac-484b-bb6a-c476f7904d27 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
26732025-01-08T18:32:14.633ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
26742025-01-08T18:32:14.633ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_0ac8ff68-15ea-4477-a206-5a7619973e85
26752025-01-08T18:32:14.651ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
26762025-01-08T18:32:14.651ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_4f399814-e4ff-4480-aa92-c4c936f90fd0
26772025-01-08T18:32:14.670ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
26782025-01-08T18:32:14.670ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_bf665b71-2255-48d6-b1fd-c863b4c8e1af
26792025-01-08T18:32:14.689ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44907
26802025-01-08T18:32:14.689ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
26812025-01-08T18:32:14.693ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
26822025-01-08T18:32:14.693ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_c1f40097-34cd-4b0e-8a1d-a4c6931740f6
26832025-01-08T18:32:14.712ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
26842025-01-08T18:32:14.712ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_4f399814-e4ff-4480-aa92-c4c936f90fd0
26852025-01-08T18:32:14.731ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
26862025-01-08T18:32:14.731ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_efc412bd-0718-4381-8d37-e5a204c8c3de
26872025-01-08T18:32:14.734ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
26882025-01-08T18:32:14.738ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43062
26892025-01-08T18:32:14.744ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 411 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43062 req_id = e0081369-6552-4714-bb6e-de2cb00d10bf response_code = 200 uri = /vpc-routes
26902025-01-08T18:32:14.757ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
26912025-01-08T18:32:14.757ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_c1f40097-34cd-4b0e-8a1d-a4c6931740f6
26922025-01-08T18:32:14.767ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
26932025-01-08T18:32:14.767ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 704 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43062 req_id = c0af63d6-62b8-475d-bdf4-805b62b310b0 response_code = 204 uri = /eip-gateways
26942025-01-08T18:32:14.768ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35633
26952025-01-08T18:32:14.773ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0041456a-da2f-4ef8-bdf7-8e18ca9d8279, kind: Service { id: 1c3a7513-ed19-483c-872f-3377ba80f37f }, name: Name("external-dns-1c3a7513-ed19-483c-872f-3377ba80f37f"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 154, 170])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ce1a1751-7375-4744-b6f0-690b02866c95, kind: Service { id: 3963284d-f267-448b-bfbb-a55402c7ef81 }, name: Name("external-dns-3963284d-f267-448b-bfbb-a55402c7ef81"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 116])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2beb6270-a9c4-41a0-b3ab-eeff6ddd3582, kind: Service { id: cef4a321-387e-44a1-92c9-23673073cdd8 }, name: Name("nexus-cef4a321-387e-44a1-92c9-23673073cdd8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 113])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bfb8fa95-f4f1-4ec3-bb77-0e8adbc6ac68, kind: Service { id: a28f8263-3fdd-4cd1-a79d-6ef15080a80e }, name: Name("nexus-a28f8263-3fdd-4cd1-a79d-6ef15080a80e"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c656ac3d-ffdc-434e-863c-8efcfbd82ab4, kind: Service { id: 099ecd74-9dbb-4a7f-94ce-79177e3bb646 }, name: Name("nexus-099ecd74-9dbb-4a7f-94ce-79177e3bb646"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 171])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
26962025-01-08T18:32:14.773ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
26972025-01-08T18:32:14.773ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
26982025-01-08T18:32:14.773ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
26992025-01-08T18:32:14.773ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27002025-01-08T18:32:14.773ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27012025-01-08T18:32:14.773ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27022025-01-08T18:32:14.773ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 832 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35633 req_id = 63b3d07d-282d-452c-8bb9-7dc74526793f response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
27032025-01-08T18:32:14.777ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27042025-01-08T18:32:14.777ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_91e62592-6bd8-45a2-a311-bfef0a5977f4
27052025-01-08T18:32:14.783ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 836 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43062 req_id = 556be626-e875-446c-92e1-861bddfccf72 response_code = 204 uri = /vpc-routes
27062025-01-08T18:32:14.794ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27072025-01-08T18:32:14.794ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_efc412bd-0718-4381-8d37-e5a204c8c3de
27082025-01-08T18:32:14.798ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46004
27092025-01-08T18:32:14.816ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
27102025-01-08T18:32:14.817ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 127860 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:44907 req_id = 73802b9d-acb3-4289-a71b-c5b77fd0bcfa response_code = 204 uri = /switch-ports
27112025-01-08T18:32:14.820ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27122025-01-08T18:32:14.820ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_91e62592-6bd8-45a2-a311-bfef0a5977f4
27132025-01-08T18:32:14.825ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4330 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44907 req_id = cb1133c5-3b1f-4052-be9e-43615801e330 response_code = 200 uri = /network-bootstore-config
27142025-01-08T18:32:14.836ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27152025-01-08T18:32:14.836ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_93dafc63-6f00-46e6-a292-a8413d960ce4
27162025-01-08T18:32:14.852ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27172025-01-08T18:32:14.853ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_93dafc63-6f00-46e6-a292-a8413d960ce4
27182025-01-08T18:32:14.857ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
27192025-01-08T18:32:14.867ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27202025-01-08T18:32:14.867ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_36139540-6634-4f0c-9c90-f57efa2432be
27212025-01-08T18:32:14.872ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54219
27222025-01-08T18:32:14.881ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
27232025-01-08T18:32:14.881ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
27242025-01-08T18:32:14.881ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/config/omicron-zones.json file = common/src/ledger.rs:177
27252025-01-08T18:32:14.890ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50059
27262025-01-08T18:32:14.890ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/config/omicron-zones.json file = common/src/ledger.rs:177
27272025-01-08T18:32:14.890ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 359850 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63516 req_id = 55075c25-cc52-46dd-8813-d6a20284854b response_code = 200 uri = /inventory
27282025-01-08T18:32:14.891ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 307 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50059 req_id = ff2fd29f-b54b-4ffc-a0dc-07b87335cdad response_code = 200 uri = /v2p
27292025-01-08T18:32:14.894ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27302025-01-08T18:32:14.894ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_36139540-6634-4f0c-9c90-f57efa2432be
27312025-01-08T18:32:14.916ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 176aea4c-4e3f-4a47-be4b-83fd0b3cc21f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27322025-01-08T18:32:14.916ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_176aea4c-4e3f-4a47-be4b-83fd0b3cc21f
27332025-01-08T18:32:14.928ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 130384 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:46004 req_id = c6d8d386-10ac-4b8a-9fa4-449d5bfda08e response_code = 204 uri = /switch-ports
27342025-01-08T18:32:14.929ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46004 req_id = 35de6963-0e77-49d6-a0b8-7e41cc4d0597 response_code = 200 uri = /network-bootstore-config
27352025-01-08T18:32:14.942ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 370216 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49171 req_id = 70b25138-2ed6-4398-8d1c-f96621fcbfa7 response_code = 200 uri = /inventory
27362025-01-08T18:32:14.950ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0ac8ff68-15ea-4477-a206-5a7619973e85 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27372025-01-08T18:32:14.950ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_0ac8ff68-15ea-4477-a206-5a7619973e85
27382025-01-08T18:32:14.976ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: bf665b71-2255-48d6-b1fd-c863b4c8e1af (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27392025-01-08T18:32:14.976ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_bf665b71-2255-48d6-b1fd-c863b4c8e1af
27402025-01-08T18:32:15.001ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46732
27412025-01-08T18:32:15.001ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0041456a-da2f-4ef8-bdf7-8e18ca9d8279, kind: Service { id: 1c3a7513-ed19-483c-872f-3377ba80f37f }, name: Name("external-dns-1c3a7513-ed19-483c-872f-3377ba80f37f"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 154, 170])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ce1a1751-7375-4744-b6f0-690b02866c95, kind: Service { id: 3963284d-f267-448b-bfbb-a55402c7ef81 }, name: Name("external-dns-3963284d-f267-448b-bfbb-a55402c7ef81"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 139, 116])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 2beb6270-a9c4-41a0-b3ab-eeff6ddd3582, kind: Service { id: cef4a321-387e-44a1-92c9-23673073cdd8 }, name: Name("nexus-cef4a321-387e-44a1-92c9-23673073cdd8"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 182, 113])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bfb8fa95-f4f1-4ec3-bb77-0e8adbc6ac68, kind: Service { id: a28f8263-3fdd-4cd1-a79d-6ef15080a80e }, name: Name("nexus-a28f8263-3fdd-4cd1-a79d-6ef15080a80e"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 239, 87])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: c656ac3d-ffdc-434e-863c-8efcfbd82ab4, kind: Service { id: 099ecd74-9dbb-4a7f-94ce-79177e3bb646 }, name: Name("nexus-099ecd74-9dbb-4a7f-94ce-79177e3bb646"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 168, 171])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
27422025-01-08T18:32:15.001ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27432025-01-08T18:32:15.001ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27442025-01-08T18:32:15.001ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
27452025-01-08T18:32:15.001ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27462025-01-08T18:32:15.001ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27472025-01-08T18:32:15.001ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27482025-01-08T18:32:15.001ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 473 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:46732 req_id = 0a660128-08cc-4e38-acf4-76e2c3629620 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
27492025-01-08T18:32:15.007ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 4f399814-e4ff-4480-aa92-c4c936f90fd0 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27502025-01-08T18:32:15.008ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_4f399814-e4ff-4480-aa92-c4c936f90fd0
27512025-01-08T18:32:15.029ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: c1f40097-34cd-4b0e-8a1d-a4c6931740f6 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27522025-01-08T18:32:15.029ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_c1f40097-34cd-4b0e-8a1d-a4c6931740f6
27532025-01-08T18:32:15.056ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: efc412bd-0718-4381-8d37-e5a204c8c3de (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27542025-01-08T18:32:15.056ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_efc412bd-0718-4381-8d37-e5a204c8c3de
27552025-01-08T18:32:15.083ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 91e62592-6bd8-45a2-a311-bfef0a5977f4 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27562025-01-08T18:32:15.083ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_91e62592-6bd8-45a2-a311-bfef0a5977f4
27572025-01-08T18:32:15.111ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 93dafc63-6f00-46e6-a292-a8413d960ce4 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27582025-01-08T18:32:15.111ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_93dafc63-6f00-46e6-a292-a8413d960ce4
27592025-01-08T18:32:15.135ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 36139540-6634-4f0c-9c90-f57efa2432be (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27602025-01-08T18:32:15.135ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_36139540-6634-4f0c-9c90-f57efa2432be
27612025-01-08T18:32:15.148ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 272942 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54219 req_id = d40329c9-e604-4482-b1fc-aa4476558f0d response_code = 200 uri = /inventory
27622025-01-08T18:32:25.663ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8721 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = eccdb9f5-d2da-4aea-8068-da682218a8a2 response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
27632025-01-08T18:32:32.458ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:42266
27642025-01-08T18:32:32.458ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:42266 req_id = 51319f80-f325-4452-9f94-2111d8828bcb response_code = 200 uri = /baseboard
27652025-01-08T18:32:44.018ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48115
27662025-01-08T18:32:44.019ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48115 req_id = 45956c0b-4007-42b4-ada3-232f4ee2a0c3 response_code = 200 uri = /vpc-routes
27672025-01-08T18:32:44.065ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47959
27682025-01-08T18:32:44.065ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47959 req_id = 2bb6fe8d-783b-4fa4-b5c4-c5d2274336af response_code = 200 uri = /vpc-routes
27692025-01-08T18:32:44.072ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
27702025-01-08T18:32:44.072ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 66 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48115 req_id = 1d283853-eb28-48ce-a81a-7dfcad5e9f36 response_code = 204 uri = /eip-gateways
27712025-01-08T18:32:44.081ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 180 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48115 req_id = 121d8e72-b6aa-4ea0-b442-5fcec16713e2 response_code = 204 uri = /vpc-routes
27722025-01-08T18:32:44.109ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
27732025-01-08T18:32:44.109ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 43 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47959 req_id = a97febea-8429-4896-9f65-dcff0e5e228b response_code = 204 uri = /eip-gateways
27742025-01-08T18:32:44.141ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48020
27752025-01-08T18:32:44.141ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48020 req_id = 2c7015d1-6c26-474c-91fd-00f397f35aff response_code = 200 uri = /v2p
27762025-01-08T18:32:44.144ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 152 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47959 req_id = ddcc952f-8750-418c-92f2-e3b73fd32204 response_code = 204 uri = /vpc-routes
27772025-01-08T18:32:44.150ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52016
27782025-01-08T18:32:44.150ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
27792025-01-08T18:32:44.188ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
27802025-01-08T18:32:44.246ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95593 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:52016 req_id = f0dcfeb3-3bac-45ff-b669-7ae51c5a088e response_code = 204 uri = /switch-ports
27812025-01-08T18:32:44.246ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40932
27822025-01-08T18:32:44.246ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34320
27832025-01-08T18:32:44.246ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34320 req_id = 2ad3471a-158a-43c1-9c3a-c40cb0f07c60 response_code = 200 uri = /v2p
27842025-01-08T18:32:44.246ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
27852025-01-08T18:32:44.252ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 781 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52016 req_id = 03ab83e5-b824-40a8-8209-cfda4500ba35 response_code = 200 uri = /network-bootstore-config
27862025-01-08T18:32:44.291ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
27872025-01-08T18:32:44.375ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 128253 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:40932 req_id = 4c94b10e-9600-4774-84cc-c5457ad9381c response_code = 204 uri = /switch-ports
27882025-01-08T18:32:44.375ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 158 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40932 req_id = 663fc1c5-f8a1-4192-985b-8e6cf7856385 response_code = 200 uri = /network-bootstore-config
27892025-01-08T18:32:44.492ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55801
27902025-01-08T18:32:44.492ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55801 req_id = 9eb56040-dfe3-4c56-9b18-505f4628e855 response_code = 200 uri = /vpc-routes
27912025-01-08T18:32:44.518ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36976
27922025-01-08T18:32:44.518ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36976 req_id = 8fe3ca78-cd7f-4519-abd0-991de5caa69f response_code = 200 uri = /v2p
27932025-01-08T18:32:44.558ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
27942025-01-08T18:32:44.558ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:55801 req_id = 36bddb13-dea3-4e1a-94b2-ffb06001f7db response_code = 204 uri = /eip-gateways
27952025-01-08T18:32:44.569ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 356 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:55801 req_id = 3e9052ae-e649-42d7-a318-56587254d201 response_code = 204 uri = /vpc-routes
27962025-01-08T18:32:44.589ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54507
27972025-01-08T18:32:44.589ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
27982025-01-08T18:32:44.630ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
27992025-01-08T18:32:44.689ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99470 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:54507 req_id = 5e75f4c1-e412-4531-b2ac-95511bba93ac response_code = 204 uri = /switch-ports
28002025-01-08T18:32:44.689ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54507 req_id = d911d921-341d-432a-bc9c-5d568ba904ec response_code = 200 uri = /network-bootstore-config
28012025-01-08T18:32:54.751ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56049
28022025-01-08T18:32:54.756ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:555 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 7f507667-dc20-4dd9-94de-947332f56b61, kind: Instance { id: c870e08e-1bec-4fba-895f-fa36e6afe266 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10963313), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.1.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.1.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 7f507667-dc20-4dd9-94de-947332f56b61, kind: Instance { id: c870e08e-1bec-4fba-895f-fa36e6afe266 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10963313), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 7f507667-dc20-4dd9-94de-947332f56b61, kind: Instance { id: c870e08e-1bec-4fba-895f-fa36e6afe266 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10963313), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(10963313))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 7f507667-dc20-4dd9-94de-947332f56b61, kind: Instance { id: c870e08e-1bec-4fba-895f-fa36e6afe266 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10963313), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [InstanceDisk { disk_id: 7402651c-9c33-4a07-b494-e958077e582a, slot: 0, read_only: false, vcr_json: .., name: "disk-a918b3c6b6" }], boot_settings: Some(InstanceBootSettings { order: [7402651c-9c33-4a07-b494-e958077e582a] }), cloud_init_bytes: Some(..) } instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 metadata = InstanceMetadata { silo_id: 6b81e3a1-bff6-48cd-81eb-794aefeaef05, project_id: bde2e64d-c834-492d-aa93-7f1d47722a7f } migration_id = None propolis_addr = [fd00:1122:3344:101::1:0]:12400 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2025-01-08T18:32:54.751647160Z }
28032025-01-08T18:32:54.756ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:588 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 migration_id = None propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28042025-01-08T18:32:54.756ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:1562 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 migration_id = None propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 7f507667-dc20-4dd9-94de-947332f56b61, kind: Instance { id: c870e08e-1bec-4fba-895f-fa36e6afe266 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10963313), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.1.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.1.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 7f507667-dc20-4dd9-94de-947332f56b61, kind: Instance { id: c870e08e-1bec-4fba-895f-fa36e6afe266 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10963313), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 7f507667-dc20-4dd9-94de-947332f56b61, kind: Instance { id: c870e08e-1bec-4fba-895f-fa36e6afe266 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10963313), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(10963313))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 7f507667-dc20-4dd9-94de-947332f56b61, kind: Instance { id: c870e08e-1bec-4fba-895f-fa36e6afe266 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10963313), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [InstanceDisk { disk_id: 7402651c-9c33-4a07-b494-e958077e582a, slot: 0, read_only: false, vcr_json: .., name: "disk-a918b3c6b6" }], boot_settings: Some(InstanceBootSettings { order: [7402651c-9c33-4a07-b494-e958077e582a] }), cloud_init_bytes: Some(..) }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2025-01-08T18:32:54.751647160Z }, propolis_addr: [fd00:1122:3344:101::1:0]:12400, migration_id: None }
28052025-01-08T18:32:54.757ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46875
28062025-01-08T18:32:54.757ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 182 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46875 req_id = 09736139-00b6-4ac6-a0cf-32b2584f104a response_code = 200 uri = /v2p
28072025-01-08T18:32:54.757ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5563 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56049 req_id = 6e607f81-8730-426b-bc71-def7c7973242 response_code = 200 uri = /vmms/997a3761-9c33-4829-a2d0-f079a98e188c
28082025-01-08T18:32:54.757ZINFOSledAgent (PortManager): Mapping virtual NIC to physical host file = illumos-utils/src/opte/port_manager.rs:874 mapping = VirtualNetworkInterfaceHost { virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), physical_host_ip: fd00:1122:3344:101::1, vni: Vni(10963313) }
28092025-01-08T18:32:54.757ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:46875 req_id = 59239275-c88a-4dab-9fd9-95bd720b0e5b response_code = 204 uri = /v2p
28102025-01-08T18:32:54.837ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54722
28112025-01-08T18:32:54.840ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2863 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:54722 req_id = bf744500-dbbe-42d5-98f5-748ec56cd384 response_code = 200 uri = /vmms/997a3761-9c33-4829-a2d0-f079a98e188c/state
28122025-01-08T18:32:54.841ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte6", ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 240, 0, 0]), slot: 0, vni: Vni { inner: 10963313 }, subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 } }) }
28132025-01-08T18:32:54.841ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
28142025-01-08T18:32:54.881ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c file = illumos-utils/src/zone.rs:325 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28152025-01-08T18:32:54.905ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c file = illumos-utils/src/zone.rs:363 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28162025-01-08T18:32:54.919ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63970
28172025-01-08T18:32:54.919ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63970 req_id = 3b14187e-991d-426d-aa68-a018e11062f8 response_code = 200 uri = /vpc-routes
28182025-01-08T18:32:54.966ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {7f507667-dc20-4dd9-94de-947332f56b61: {10.151.1.175: {b78022a3-2a95-4330-a9a6-7d964f3c6a88}, 10.151.1.176: {b78022a3-2a95-4330-a9a6-7d964f3c6a88}}} }
28192025-01-08T18:32:55.662ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7565 local_addr = [fd00:1122:3344:101::1]:33327 method = GET remote_addr = [fd00:1122:3344:101::d]:43465 req_id = 89ed5b70-4efe-4e0b-b9fd-91e00f301cf5 response_code = 200 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 uri = /7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
28202025-01-08T18:32:56.372ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlInstance0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1:0'/> <propval type="astring" name="listen_port" value='12400'/> <propval type="astring" name="metric_addr" value='dns'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28212025-01-08T18:32:56.372ZINFOSledAgent (InstanceManager): Booting oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c zone file = illumos-utils/src/running_zone.rs:488 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c zone = oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c
28222025-01-08T18:32:56.548ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c") failed: Property not found. retry in 29.621861ms file = illumos-utils/src/svc.rs:36 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c zone = oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c
28232025-01-08T18:32:56.587ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c") failed: Property not found. retry in 82.266629ms file = illumos-utils/src/svc.rs:36 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c zone = oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c
28242025-01-08T18:32:56.684ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c") failed: Property not found. retry in 143.701024ms file = illumos-utils/src/svc.rs:36 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c zone = oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c
28252025-01-08T18:32:56.845ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c") failed: Property not found. retry in 258.506024ms file = illumos-utils/src/svc.rs:36 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c zone = oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c
28262025-01-08T18:32:57.118ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c") failed: Property not found. retry in 412.018236ms file = illumos-utils/src/svc.rs:36 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c zone = oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c
28272025-01-08T18:32:57.541ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c") failed: Property not found. retry in 848.385956ms file = illumos-utils/src/svc.rs:36 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c zone = oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c
28282025-01-08T18:32:58.401ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c") failed: Property not found. retry in 585.902811ms file = illumos-utils/src/svc.rs:36 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c zone = oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c
28292025-01-08T18:32:58.998ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c") failed: Property not found. retry in 908.239983ms file = illumos-utils/src/svc.rs:36 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c zone = oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c
28302025-01-08T18:32:59.917ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c") failed: Property not found. retry in 1.089448897s file = illumos-utils/src/svc.rs:36 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c zone = oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c
28312025-01-08T18:33:01.018ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c") failed: Property not found. retry in 1.317865957s file = illumos-utils/src/svc.rs:36 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c zone = oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c
28322025-01-08T18:33:02.366ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c file = sled-agent/src/instance.rs:2067 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28332025-01-08T18:33:02.375ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:2076 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28342025-01-08T18:33:02.413ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:2100 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28352025-01-08T18:33:02.491ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 remote_addr = [fdb0:18c0:4d0d:9fb2::2]:32796
28362025-01-08T18:33:02.491ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41 local_addr = [fdb0:18c0:4d0d:9fb2::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:9fb2::2]:32796 req_id = 043ba5a9-aba4-426c-a6e5-b4af080ed4fc response_code = 200 uri = /baseboard
28372025-01-08T18:33:02.555ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:1213 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28382025-01-08T18:33:02.556ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1249 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
28392025-01-08T18:33:02.556ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Creating), migration_in: None, migration_out: None, time: 2025-01-08T18:33:02.556592636Z } file = sled-agent/src/instance.rs:780 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28402025-01-08T18:33:02.556ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7590620 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63970 req_id = 3041bd34-77b6-4a3e-b438-cf3746cd8f4b response_code = 204 uri = /eip-gateways
28412025-01-08T18:33:02.556ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2025-01-08T18:33:02.556592636Z } propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28422025-01-08T18:33:02.556ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2025-01-08T18:33:02.556592636Z }, migration_in: None, migration_out: None }
28432025-01-08T18:33:02.607ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_in: None, migration_out: None, time: 2025-01-08T18:33:02.607810133Z } file = sled-agent/src/instance.rs:780 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28442025-01-08T18:33:02.607ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-01-08T18:33:02.607810133Z } propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28452025-01-08T18:33:02.607ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-01-08T18:33:02.607810133Z }, migration_in: None, migration_out: None }
28462025-01-08T18:33:02.701ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 504 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63970 req_id = 291341e6-b48a-4f48-b8a8-8f979aed4f1e response_code = 204 uri = /vpc-routes
28472025-01-08T18:33:12.284ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57902
28482025-01-08T18:33:12.284ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 558 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:57902 req_id = bee9b026-5a70-40ae-a207-12cf276e21df response_code = 200 uri = /vmms/997a3761-9c33-4829-a2d0-f079a98e188c/state
28492025-01-08T18:33:12.284ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_in: None, migration_out: None, time: 2025-01-08T18:33:12.284767308Z } file = sled-agent/src/instance.rs:780 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28502025-01-08T18:33:12.284ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2025-01-08T18:33:12.284767308Z } propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28512025-01-08T18:33:12.284ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c state = SledVmmState { vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2025-01-08T18:33:12.284767308Z }, migration_in: None, migration_out: None }
28522025-01-08T18:33:12.336ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_in: None, migration_out: None, time: 2025-01-08T18:33:12.336707765Z } file = sled-agent/src/instance.rs:780 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28532025-01-08T18:33:12.336ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2025-01-08T18:33:12.336707765Z } propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28542025-01-08T18:33:12.336ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:816 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28552025-01-08T18:33:12.336ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
28562025-01-08T18:33:12.337ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
28572025-01-08T18:33:12.337ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/74c5b5be-8c07-4870-ad96-3bd00809d0ec/debug/bundle/zone", "/pool/int/e4d4e46e-f335-4351-a367-c84fc95a9cd2/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/176aea4c-4e3f-4a47-be4b-83fd0b3cc21f/crypt/debug", "/pool/ext/0ac8ff68-15ea-4477-a206-5a7619973e85/crypt/debug", "/pool/ext/bf665b71-2255-48d6-b1fd-c863b4c8e1af/crypt/debug", "/pool/ext/4f399814-e4ff-4480-aa92-c4c936f90fd0/crypt/debug", "/pool/ext/c1f40097-34cd-4b0e-8a1d-a4c6931740f6/crypt/debug", "/pool/ext/efc412bd-0718-4381-8d37-e5a204c8c3de/crypt/debug", "/pool/ext/91e62592-6bd8-45a2-a311-bfef0a5977f4/crypt/debug", "/pool/ext/93dafc63-6f00-46e6-a292-a8413d960ce4/crypt/debug", "/pool/ext/36139540-6634-4f0c-9c90-f57efa2432be/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:339 zone_name = oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c
28582025-01-08T18:33:12.969ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1142 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c", bundle_id: a029e053-691f-4d16-9559-6bcb613028b7 }, time_created: 2025-01-08T18:33:12.338006659Z, version: 0, cause: TerminatedInstance }
28592025-01-08T18:33:12.969ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_997a3761-9c33-4829-a2d0-f079a98e188c file = sled-agent/src/instance.rs:1313 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
28602025-01-08T18:33:14.140ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58366
28612025-01-08T18:33:14.207ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51233
28622025-01-08T18:33:14.214ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53521
28632025-01-08T18:33:14.214ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53521 req_id = 5e2c1b71-3b61-41ed-b286-f047d082f444 response_code = 200 uri = /vpc-routes
28642025-01-08T18:33:14.266ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45696
28652025-01-08T18:33:14.266ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45696 req_id = f226929d-5f71-4e46-913b-dc623e5c7fb1 response_code = 200 uri = /vpc-routes
28662025-01-08T18:33:14.272ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {7f507667-dc20-4dd9-94de-947332f56b61: {10.151.1.176: {b78022a3-2a95-4330-a9a6-7d964f3c6a88}, 10.151.1.175: {b78022a3-2a95-4330-a9a6-7d964f3c6a88}}} }
28672025-01-08T18:33:14.272ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 160 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53521 req_id = e2878e64-44ef-4879-a2d4-78eb137557d5 response_code = 204 uri = /eip-gateways
28682025-01-08T18:33:14.336ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 389 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53521 req_id = 6b9aaa35-195e-4dc0-a2ef-1543054f7b83 response_code = 204 uri = /vpc-routes
28692025-01-08T18:33:14.381ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {7f507667-dc20-4dd9-94de-947332f56b61: {10.151.1.175: {b78022a3-2a95-4330-a9a6-7d964f3c6a88}, 10.151.1.176: {b78022a3-2a95-4330-a9a6-7d964f3c6a88}}} }
28702025-01-08T18:33:14.381ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45696 req_id = 8359eedd-000f-4571-8520-e6c3103c65e8 response_code = 204 uri = /eip-gateways
28712025-01-08T18:33:14.415ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 253 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45696 req_id = 6e09a8d1-d2e5-4bb2-89e9-6aae9089e545 response_code = 204 uri = /vpc-routes
28722025-01-08T18:33:14.468ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37439
28732025-01-08T18:33:14.468ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
28742025-01-08T18:33:14.512ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
28752025-01-08T18:33:14.580ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111899 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:37439 req_id = bbdb7831-429c-468b-b8e0-60130e0cddf3 response_code = 204 uri = /switch-ports
28762025-01-08T18:33:14.580ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53907
28772025-01-08T18:33:14.580ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51462
28782025-01-08T18:33:14.580ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46694
28792025-01-08T18:33:14.585ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 539 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53907 req_id = 5963b780-ad50-4d67-9f45-bb110a392435 response_code = 200 uri = /v2p
28802025-01-08T18:33:14.586ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5848 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37439 req_id = e18607d8-54a9-4043-9ce2-0e177118c50e response_code = 200 uri = /network-bootstore-config
28812025-01-08T18:33:14.586ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 377 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46694 req_id = 8f56bf45-ee94-4192-b896-cf5d7d03c88c response_code = 200 uri = /v2p
28822025-01-08T18:33:14.586ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
28832025-01-08T18:33:14.626ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
28842025-01-08T18:33:14.664ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64799
28852025-01-08T18:33:14.684ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98237 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:51462 req_id = e9b66e03-7cb9-4701-acb8-a3f947e768b0 response_code = 204 uri = /switch-ports
28862025-01-08T18:33:14.684ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 138 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51462 req_id = b7f00edf-fca4-493f-b654-ca20b06d2504 response_code = 200 uri = /network-bootstore-config
28872025-01-08T18:33:14.732ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42504
28882025-01-08T18:33:14.732ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42504 req_id = 946db403-75a3-4ec7-99d0-62f427212fae response_code = 200 uri = /vpc-routes
28892025-01-08T18:33:14.781ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {7f507667-dc20-4dd9-94de-947332f56b61: {10.151.1.175: {b78022a3-2a95-4330-a9a6-7d964f3c6a88}, 10.151.1.176: {b78022a3-2a95-4330-a9a6-7d964f3c6a88}}} }
28902025-01-08T18:33:14.781ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 139 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42504 req_id = 2ec4a624-ebed-41bd-9c09-4ec19680a255 response_code = 204 uri = /eip-gateways
28912025-01-08T18:33:14.797ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 276 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42504 req_id = 65e0e8d6-4ff2-4bd1-a936-a730f1743dd5 response_code = 204 uri = /vpc-routes
28922025-01-08T18:33:14.886ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48992
28932025-01-08T18:33:14.886ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
28942025-01-08T18:33:14.927ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
28952025-01-08T18:33:14.982ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96081 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:48992 req_id = 971ee0e6-6b2d-4baa-b935-9885dca17549 response_code = 204 uri = /switch-ports
28962025-01-08T18:33:14.982ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36937
28972025-01-08T18:33:14.987ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 958 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48992 req_id = e87a31c9-2c9f-483a-b6c0-e6045014b9fb response_code = 200 uri = /network-bootstore-config
28982025-01-08T18:33:14.987ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 371 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36937 req_id = 78dcdbba-2490-40f6-a8f4-07dbe1700c74 response_code = 200 uri = /v2p
28992025-01-08T18:33:14.994ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:279 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
29002025-01-08T18:33:15.010ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:1000 id = RouterId { vni: Vni(10963313), kind: System }
29012025-01-08T18:33:15.010ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:1000 id = RouterId { vni: Vni(10963313), kind: Custom(V4(Ipv4Net { addr: 172.30.0.0, width: 22 })) }
29022025-01-08T18:33:15.011ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-01-08T18:33:15.011053626Z }, migration_in: None, migration_out: None }
29032025-01-08T18:33:15.060ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-01-08T18:33:15.011053626Z }, migration_in: None, migration_out: None }
29042025-01-08T18:33:15.065ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:1250 instance_id = c870e08e-1bec-4fba-895f-fa36e6afe266 propolis_id = 997a3761-9c33-4829-a2d0-f079a98e188c
29052025-01-08T18:33:15.096ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 955744 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58366 req_id = 7a7ef813-c74d-4b91-96dc-6bfcf4371a44 response_code = 200 uri = /vmms/997a3761-9c33-4829-a2d0-f079a98e188c/state
29062025-01-08T18:33:15.096ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 888697 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51233 req_id = 8bf82cd5-f432-4eae-9c20-928f8587b953 response_code = 200 uri = /vmms/997a3761-9c33-4829-a2d0-f079a98e188c/state
29072025-01-08T18:33:15.096ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 431813 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64799 req_id = 4a0c37f6-2f11-48a2-9bcc-ed693b028875 response_code = 200 uri = /vmms/997a3761-9c33-4829-a2d0-f079a98e188c/state
29082025-01-08T18:33:15.546ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63560
29092025-01-08T18:33:15.546ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63560 req_id = 07eefce1-42bd-4c7a-8682-835b9bb26746 response_code = 200 uri = /vpc-routes
29102025-01-08T18:33:15.598ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
29112025-01-08T18:33:15.598ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1249 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0
29122025-01-08T18:33:15.598ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63560 req_id = 1c328f92-714e-463a-b1ad-9497e0d1ed0b response_code = 204 uri = /eip-gateways
29132025-01-08T18:33:15.607ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 159 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63560 req_id = 1fab4133-d9e2-4dcd-ba13-daa64faf6109 response_code = 204 uri = /vpc-routes
29142025-01-08T18:33:15.744ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42023
29152025-01-08T18:33:15.744ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42023 req_id = 7c35320d-5d1c-475b-83c9-57b570a367c9 response_code = 200 uri = /v2p
29162025-01-08T18:33:16.268ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52687
29172025-01-08T18:33:16.268ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52687 req_id = aecaccac-dc32-49a3-a75c-595385715cec response_code = 200 uri = /v2p
29182025-01-08T18:33:16.387ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35520
29192025-01-08T18:33:16.387ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 34 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35520 req_id = c566b742-6441-4f80-942d-8643bab89412 response_code = 200 uri = /vpc-routes
29202025-01-08T18:33:16.402ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
29212025-01-08T18:33:16.402ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 50 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35520 req_id = 9ee80bd1-5545-4b65-9b31-59bc5b0ec644 response_code = 204 uri = /eip-gateways
29222025-01-08T18:33:16.410ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35520 req_id = 38cc81e1-6d79-4f3d-8f8f-e5997439747e response_code = 204 uri = /vpc-routes
29232025-01-08T18:33:16.996ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37812
29242025-01-08T18:33:16.996ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 71 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37812 req_id = 12fe5057-4b3a-45ec-b5c4-4e6ff91ca9a1 response_code = 200 uri = /vpc-routes
29252025-01-08T18:33:17.038ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 7d4a0e1a-01d9-4e26-ba9c-4e51baa26ea0 values = ExternalIpGatewayMap { mappings: {} }
29262025-01-08T18:33:17.038ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37812 req_id = 2c33ea75-5dd0-450b-88ee-596561df63b9 response_code = 204 uri = /eip-gateways
29272025-01-08T18:33:17.046ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 163 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37812 req_id = d63d427f-b756-4d82-8046-14a2957d89d7 response_code = 204 uri = /vpc-routes