Max level shown:
1 [ Jan 10 21:54:26 Enabled. ]
2 [ Jan 10 21:54:26 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Jan 10 21:54:26 Method "start" exited with status 0. ]
4 [ Jan 10 21:54:26 Rereading configuration. ]
5 [ Jan 10 21:54:26 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72025-01-10T21:54:26.691ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:113
82025-01-10T21:54:26.691ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:255
92025-01-10T21:54:26.744ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:262
102025-01-10T21:54:26.744ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
112025-01-10T21:54:26.859ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
122025-01-10T21:54:26.868ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
132025-01-10T21:54:26.877ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:272 path = /var/run/oxide/
142025-01-10T21:54:27.479ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fdb0:18c0:4d0d:a02a::, width: 64 }
152025-01-10T21:54:27.618ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:126
162025-01-10T21:54:27.618ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:138
172025-01-10T21:54:27.618ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:151
182025-01-10T21:54:27.618ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:173 nongimlet_observed_disks = [] sled_mode = Scrimlet { asic: SoftNpuZone }
192025-01-10T21:54:27.618ZINFOSledAgent (DumpSetup-worker): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:520
202025-01-10T21:54:27.618ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:717
212025-01-10T21:54:27.621ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "sock", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:624
222025-01-10T21:54:27.621ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:766
232025-01-10T21:54:27.621ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:187
242025-01-10T21:54:27.621ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:825
252025-01-10T21:54:27.621ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_0.vdev
262025-01-10T21:54:27.621ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = None
272025-01-10T21:54:27.621ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
282025-01-10T21:54:27.621ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
292025-01-10T21:54:27.621ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } path = /scratch/m2_0.vdev
302025-01-10T21:54:27.633ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_0.vdev file = sled-hardware/src/disk.rs:367
312025-01-10T21:54:27.633ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = 53b46f94-5cb9-4bbb-a551-b5457accb3ee (zpool)
322025-01-10T21:54:27.733ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 53b46f94-5cb9-4bbb-a551-b5457accb3ee (zpool), kind: Internal }
332025-01-10T21:54:28.083ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:607
342025-01-10T21:54:28.083ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_1.vdev
352025-01-10T21:54:28.083ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
362025-01-10T21:54:28.083ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
372025-01-10T21:54:28.083ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
382025-01-10T21:54:28.083ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
392025-01-10T21:54:28.083ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } path = /scratch/m2_1.vdev
402025-01-10T21:54:28.092ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_1.vdev file = sled-hardware/src/disk.rs:367
412025-01-10T21:54:28.092ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = 10481327-13d1-4548-950f-130d063e2690 (zpool)
422025-01-10T21:54:28.175ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 10481327-13d1-4548-950f-130d063e2690 (zpool), kind: Internal }
432025-01-10T21:54:28.529ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_0.vdev
442025-01-10T21:54:28.529ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 53b46f94-5cb9-4bbb-a551-b5457accb3ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 10481327-13d1-4548-950f-130d063e2690 (zpool), kind: Internal } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
452025-01-10T21:54:28.529ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
462025-01-10T21:54:28.529ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
472025-01-10T21:54:28.529ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
482025-01-10T21:54:28.529ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
492025-01-10T21:54:28.529ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_1.vdev
502025-01-10T21:54:28.529ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 53b46f94-5cb9-4bbb-a551-b5457accb3ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 10481327-13d1-4548-950f-130d063e2690 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
512025-01-10T21:54:28.529ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
522025-01-10T21:54:28.529ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
532025-01-10T21:54:28.529ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
542025-01-10T21:54:28.529ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
552025-01-10T21:54:28.529ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_2.vdev
562025-01-10T21:54:28.529ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 53b46f94-5cb9-4bbb-a551-b5457accb3ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 10481327-13d1-4548-950f-130d063e2690 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
572025-01-10T21:54:28.529ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
582025-01-10T21:54:28.529ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
592025-01-10T21:54:28.529ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
602025-01-10T21:54:28.529ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
612025-01-10T21:54:28.529ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_3.vdev
622025-01-10T21:54:28.529ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
632025-01-10T21:54:28.529ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
642025-01-10T21:54:28.529ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 53b46f94-5cb9-4bbb-a551-b5457accb3ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 10481327-13d1-4548-950f-130d063e2690 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
652025-01-10T21:54:28.529ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
662025-01-10T21:54:28.529ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
672025-01-10T21:54:28.529ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_4.vdev
682025-01-10T21:54:28.529ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
692025-01-10T21:54:28.529ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
702025-01-10T21:54:28.529ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 53b46f94-5cb9-4bbb-a551-b5457accb3ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 10481327-13d1-4548-950f-130d063e2690 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
712025-01-10T21:54:28.530ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_5.vdev
722025-01-10T21:54:28.530ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
732025-01-10T21:54:28.530ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
742025-01-10T21:54:28.530ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
752025-01-10T21:54:28.530ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
762025-01-10T21:54:28.530ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 53b46f94-5cb9-4bbb-a551-b5457accb3ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 10481327-13d1-4548-950f-130d063e2690 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
772025-01-10T21:54:28.530ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
782025-01-10T21:54:28.530ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
792025-01-10T21:54:28.530ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_6.vdev
802025-01-10T21:54:28.530ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 53b46f94-5cb9-4bbb-a551-b5457accb3ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 10481327-13d1-4548-950f-130d063e2690 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
812025-01-10T21:54:28.530ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
822025-01-10T21:54:28.530ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
832025-01-10T21:54:28.530ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
842025-01-10T21:54:28.530ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
852025-01-10T21:54:28.530ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_7.vdev
862025-01-10T21:54:28.530ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 53b46f94-5cb9-4bbb-a551-b5457accb3ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 10481327-13d1-4548-950f-130d063e2690 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
872025-01-10T21:54:28.530ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
882025-01-10T21:54:28.530ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
892025-01-10T21:54:28.530ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
902025-01-10T21:54:28.530ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
912025-01-10T21:54:28.530ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_8.vdev
922025-01-10T21:54:28.530ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 53b46f94-5cb9-4bbb-a551-b5457accb3ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 10481327-13d1-4548-950f-130d063e2690 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
932025-01-10T21:54:28.530ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
942025-01-10T21:54:28.530ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
952025-01-10T21:54:28.530ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
962025-01-10T21:54:28.530ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
972025-01-10T21:54:28.530ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:97
982025-01-10T21:54:28.530ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-agent/src/long_running_tasks.rs:99
992025-01-10T21:54:28.530ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 53b46f94-5cb9-4bbb-a551-b5457accb3ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 10481327-13d1-4548-950f-130d063e2690 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
1002025-01-10T21:54:28.530ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
1012025-01-10T21:54:28.530ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
1022025-01-10T21:54:28.530ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1032025-01-10T21:54:28.530ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:211
1042025-01-10T21:54:28.530ZINFOSledAgent (bootstore): No ledger in /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1052025-01-10T21:54:28.530ZINFOSledAgent (bootstore): No ledger in /pool/int/10481327-13d1-4548-950f-130d063e2690/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1062025-01-10T21:54:28.530ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-sock-i86pc
1072025-01-10T21:54:28.530ZINFOSledAgent (bootstore): No ledger in /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1082025-01-10T21:54:28.530ZINFOSledAgent (bootstore): No ledger in /pool/int/10481327-13d1-4548-950f-130d063e2690/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1092025-01-10T21:54:28.530ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-sock-i86pc
1102025-01-10T21:54:28.530ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:216
1112025-01-10T21:54:28.530ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:231
1122025-01-10T21:54:28.540ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:748
1132025-01-10T21:54:28.540ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 3133, tv_nsec: 759730291 } time_to_next_cleanup = 599.999926668s
1142025-01-10T21:54:28.540ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3973
1152025-01-10T21:54:28.540ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
1162025-01-10T21:54:28.540ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/sled-agent-request.json file = common/src/ledger.rs:188
1172025-01-10T21:54:28.541ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/10481327-13d1-4548-950f-130d063e2690/config/sled-agent-request.json file = common/src/ledger.rs:188
1182025-01-10T21:54:28.544ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fdb0:18c0:4d0d:a02a::1]:80
1192025-01-10T21:54:28.546ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:51 local_addr = [fdb0:18c0:4d0d:a02a::1]:12346
1202025-01-10T21:54:28.546ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, rib_priority: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: Some( None, ), bgp_peers: [], autoneg: false, lldp: None, tx_eq: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, } file = sled-agent/src/rack_setup/service.rs:1204
1212025-01-10T21:54:28.546ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1222025-01-10T21:54:28.546ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
1232025-01-10T21:54:28.546ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
1242025-01-10T21:54:28.546ZINFOSledAgent (RSS): No ledger in /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/rss-started.marker file = common/src/ledger.rs:188
1252025-01-10T21:54:28.546ZINFOSledAgent (RSS): No ledger in /pool/int/10481327-13d1-4548-950f-130d063e2690/config/rss-started.marker file = common/src/ledger.rs:188
1262025-01-10T21:54:28.546ZINFOSledAgent (RSS): No ledger in /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1272025-01-10T21:54:28.546ZINFOSledAgent (RSS): No ledger in /pool/int/10481327-13d1-4548-950f-130d063e2690/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1282025-01-10T21:54:28.546ZINFOSledAgent (RSS): RSS not previously run. Creating plans. file = sled-agent/src/rack_setup/service.rs:1253
1292025-01-10T21:54:28.546ZINFOSledAgent (RSS): Writing ledger to /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/.rss-started.marker.tmp file = common/src/ledger.rs:199
1302025-01-10T21:54:28.546ZINFOSledAgent (RSS): Writing ledger to /pool/int/10481327-13d1-4548-950f-130d063e2690/config/.rss-started.marker.tmp file = common/src/ledger.rs:199
1312025-01-10T21:54:28.547ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:1285
1322025-01-10T21:54:28.547ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:66
1332025-01-10T21:54:28.547ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:a02a::1 file = sled-agent/src/rack_setup/plan/sled.rs:39
1342025-01-10T21:54:28.547ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:1316
1352025-01-10T21:54:28.547ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
1362025-01-10T21:54:28.547ZINFOSledAgent (bootstore): Writing ledger to /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1372025-01-10T21:54:28.547ZINFOSledAgent (bootstore): Writing ledger to /pool/int/10481327-13d1-4548-950f-130d063e2690/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1382025-01-10T21:54:28.547ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
1392025-01-10T21:54:28.547ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:232 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 2235d715-657e-4a92-8f59-9a6c0c957b73 (sled), rack_id: 9e742e19-15eb-4edc-b3b4-60e767ee3ba8, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
1402025-01-10T21:54:28.548ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:4270
1412025-01-10T21:54:28.548ZINFOSledAgent (ServiceManager): Starting switch zone file = sled-agent/src/services.rs:4664
1422025-01-10T21:54:28.552ZINFOSledAgent (SledAgentSprocketsClient): Loaded keys and certs BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/client.rs:50
1432025-01-10T21:54:28.558ZINFOSledAgent (SledAgentSprocketsClient): Certificate chain verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1442025-01-10T21:54:28.558ZINFOSledAgent (SledAgentSprocketsClient): Signature verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1452025-01-10T21:54:28.562ZINFOSledAgent (SledAgentSprocketsServer): Certificate chain verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1462025-01-10T21:54:28.562ZINFOSledAgent (SledAgentSprocketsServer): Signature verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1472025-01-10T21:54:28.562ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:74 remote_addr = [fdb0:18c0:4d0d:a02a::1]:37010
1482025-01-10T21:54:28.563ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 2235d715-657e-4a92-8f59-9a6c0c957b73 (sled), rack_id: 9e742e19-15eb-4edc-b3b4-60e767ee3ba8, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } file = sled-agent/src/bootstrap/server.rs:362
1492025-01-10T21:54:28.563ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:377
1502025-01-10T21:54:28.563ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:44
1512025-01-10T21:54:28.563ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1522025-01-10T21:54:28.563ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:607
1532025-01-10T21:54:28.563ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:101::, width: 64 }
1542025-01-10T21:54:28.563ZINFOSledAgent (StorageManager): Loading storage config file = sled-storage/src/manager.rs:813
1552025-01-10T21:54:28.563ZINFOSledAgent (StorageManager): No ledger in /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1562025-01-10T21:54:28.563ZINFOSledAgent (StorageManager): No ledger in /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1572025-01-10T21:54:28.563ZINFOSledAgent (StorageManager): No ledger of physical disks exists file = sled-storage/src/manager.rs:802
1582025-01-10T21:54:28.563ZINFOSledAgent (StorageManager): KeyManager ready, but no ledger detected file = sled-storage/src/manager.rs:830
1592025-01-10T21:54:28.563ZINFOSledAgent (StorageManager): automatically managing disks file = sled-storage/src/manager.rs:596
1602025-01-10T21:54:28.563ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
1612025-01-10T21:54:28.563ZINFOSledAgent (StorageManager): Successfully synchronized disks without error file = sled-storage/src/manager.rs:701 result = DisksManagementResult { status: [] }
1622025-01-10T21:54:28.579ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
1632025-01-10T21:54:28.599ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:325
1642025-01-10T21:54:28.614ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:412 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
1652025-01-10T21:54:28.614ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
1662025-01-10T21:54:28.614ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:424 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
1672025-01-10T21:54:28.614ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1682025-01-10T21:54:28.616ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:363
1692025-01-10T21:54:28.620ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 68.013904ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "1f747d60-adc5-4a2f-9f9e-c5719ec013e5", "content-length": "124", "date": "Fri, 10 Jan 2025 21:54:28 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "1f747d60-adc5-4a2f-9f9e-c5719ec013e5" } file = clients/ddm-admin-client/src/lib.rs:177
1702025-01-10T21:54:28.627ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_53b46f94-5cb9-4bbb-a551-b5457accb3ee/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1712025-01-10T21:54:28.651ZINFOSledAgent: successfully created encrypted zvol: name="oxi_53b46f94-5cb9-4bbb-a551-b5457accb3ee/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1722025-01-10T21:54:28.651ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_53b46f94-5cb9-4bbb-a551-b5457accb3ee/swap" file = sled-agent/src/swap_device.rs:96
1732025-01-10T21:54:28.656ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:439 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
1742025-01-10T21:54:28.656ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:129
1752025-01-10T21:54:28.656ZINFOSledAgent (BackingFs): Ensuring dataset oxi_53b46f94-5cb9-4bbb-a551-b5457accb3ee/backing/fmd file = sled-agent/src/backing_fs.rs:139
1762025-01-10T21:54:28.690ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1772025-01-10T21:54:28.690ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 89.716195ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "98a32623-2ca7-423c-9374-230d9b64d2cb", "content-length": "124", "date": "Fri, 10 Jan 2025 21:54:28 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "98a32623-2ca7-423c-9374-230d9b64d2cb" } file = clients/ddm-admin-client/src/lib.rs:177
1782025-01-10T21:54:28.723ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:174
1792025-01-10T21:54:28.781ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1802025-01-10T21:54:28.782ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 82.802147ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "0fa5cb4f-0751-4dda-b880-df6f95fa76d1", "content-length": "124", "date": "Fri, 10 Jan 2025 21:54:28 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "0fa5cb4f-0751-4dda-b880-df6f95fa76d1" } file = clients/ddm-admin-client/src/lib.rs:177
1812025-01-10T21:54:28.866ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1822025-01-10T21:54:28.866ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 134.242272ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "e8540379-7d05-4e6e-b019-f7a1846993ec", "content-length": "124", "date": "Fri, 10 Jan 2025 21:54:28 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "e8540379-7d05-4e6e-b019-f7a1846993ec" } file = clients/ddm-admin-client/src/lib.rs:177
1832025-01-10T21:54:28.942ZINFOSledAgent (BackingFs): Mounting oxi_53b46f94-5cb9-4bbb-a551-b5457accb3ee/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:182
1842025-01-10T21:54:28.955ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:190
1852025-01-10T21:54:28.955ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:190
1862025-01-10T21:54:28.955ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:190
1872025-01-10T21:54:28.955ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:196
1882025-01-10T21:54:29.002ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1892025-01-10T21:54:29.002ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 208.979061ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "a53ba5f2-b378-4a55-b4ce-08f88fd48318", "content-length": "124", "date": "Fri, 10 Jan 2025 21:54:28 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "a53ba5f2-b378-4a55-b4ce-08f88fd48318" } file = clients/ddm-admin-client/src/lib.rs:177
1902025-01-10T21:54:29.213ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1912025-01-10T21:54:29.214ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 145.456957ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "7814b402-175e-4894-87df-10841d3eb186", "content-length": "124", "date": "Fri, 10 Jan 2025 21:54:28 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "7814b402-175e-4894-87df-10841d3eb186" } file = clients/ddm-admin-client/src/lib.rs:177
1922025-01-10T21:54:29.361ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1932025-01-10T21:54:29.362ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 410.311017ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "f187625a-d744-4d9a-9bfd-c7a094650670", "content-length": "124", "date": "Fri, 10 Jan 2025 21:54:28 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "f187625a-d744-4d9a-9bfd-c7a094650670" } file = clients/ddm-admin-client/src/lib.rs:177
1942025-01-10T21:54:29.686ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:93 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
1952025-01-10T21:54:29.691ZINFOSledAgent (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:37589 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
1962025-01-10T21:54:29.692ZINFOSledAgent (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
1972025-01-10T21:54:29.692ZINFOSledAgent (producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::1]:37589 file = oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
1982025-01-10T21:54:29.693ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 357.907894ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
1992025-01-10T21:54:29.693ZINFOSledAgent (VmmReservoirManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/vmm_reservoir.rs:267 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
2002025-01-10T21:54:29.693ZINFOSledAgent (VmmReservoirManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:275 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
2012025-01-10T21:54:29.693ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:932 underlay_address = fd00:1122:3344:101::1
2022025-01-10T21:54:29.693ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
2032025-01-10T21:54:29.773ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
2042025-01-10T21:54:30.052ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 666.054848ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
2052025-01-10T21:54:30.720ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 1.139713966s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
2062025-01-10T21:54:31.862ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 1.270868526s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
2072025-01-10T21:54:32.453ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2694
2082025-01-10T21:54:32.453ZINFOSledAgent (ServiceManager): Setting up lldpd service file = sled-agent/src/services.rs:2939
2092025-01-10T21:54:32.453ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2564
2102025-01-10T21:54:32.454ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2611
2112025-01-10T21:54:32.454ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:3020
2122025-01-10T21:54:32.454ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:3065
2132025-01-10T21:54:32.455ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2864
2142025-01-10T21:54:32.457ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2605
2152025-01-10T21:54:32.457ZINFOSledAgent (ServiceManager): Profile for oxz_switch: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/mgs"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12225'/> <propval type="astring" name="id" value='7e95cc42-722c-4dc2-8346-5afcadd03955'/> <propval type="astring" name="rack_id" value='9e742e19-15eb-4edc-b3b4-60e767ee3ba8'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/wicketd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12226'/> <propval type="astring" name="artifact-address" value='[fdb0:18c0:4d0d:a02a::2]:12227'/> <propval type="astring" name="baseboard-file" value='/opt/oxide/baseboard.json'/> <propval type="astring" name="mgs-address" value='[::1]:12225'/> <propval type="astring" name="nexus-proxy-address" value='[::]:12229'/> <propval type="astring" name="rack-subnet" value='fd00:1122:3344::'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/switch_zone_setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="gz_local_link_addr" value='fe80::8:20ff:fedd:864f'/> <property type="astring" name="link_local_links"> <astring_list> <value_node value="tfportrear0_0" /> <value_node value="oxBootstrap0" /> </astring_list> </property> <propval type="astring" name="bootstrap_addr" value='fdb0:18c0:4d0d:a02a::2'/> <propval type="astring" name="bootstrap_vnic" value='oxBootstrap0'/> <propval type="astring" name="baseboard_info" value='{"type":"pc","identifier":"sock","model":"i86pc"}'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/dendrite"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="rack_id" value='9e742e19-15eb-4edc-b3b4-60e767ee3ba8'/> <propval type="astring" name="sled_id" value='2235d715-657e-4a92-8f59-9a6c0c957b73'/> <propval type="astring" name="sled_model" value='i86pc'/> <propval type="astring" name="sled_serial" value='sock'/> <propval type="astring" name="sled_revision" value='0'/> <propval type="astring" name="address" value='[::1]:12224'/> <propval type="astring" name="front_ports" value='1'/> <propval type="astring" name="rear_ports" value='1'/> <propval type="astring" name="port_config" value='/opt/oxide/dendrite/misc/softnpu_single_sled_config.toml'/> <propval type="astring" name="mgmt" value='uds'/> <propval type="astring" name="uds_path" value='/opt/softnpu/stuff'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/tfport"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="host" value='[::1]'/> <propval type="astring" name="port" value='12224'/> <propval type="astring" name="flags" value='--sync-only'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/lldpd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="board_rev" value='softnpu_front_1_rear_1'/> <propval type="astring" name="scrimlet_id" value='sock'/> <propval type="astring" name="scrimlet_model" value='i86pc'/> <propval type="astring" name="address" value='[::1]:12230'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/pumpkind"> </service> <service version="1" type="service" name="oxide/mgd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_uuid" value='2235d715-657e-4a92-8f59-9a6c0c957b73'/> <propval type="astring" name="rack_uuid" value='9e742e19-15eb-4edc-b3b4-60e767ee3ba8'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/mg-ddm"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="mode" value='transit'/> <propval type="astring" name="dendrite" value='true'/> <propval type="astring" name="sled_uuid" value='2235d715-657e-4a92-8f59-9a6c0c957b73'/> <propval type="astring" name="rack_uuid" value='9e742e19-15eb-4edc-b3b4-60e767ee3ba8'/> <propval type="astring" name="interfaces" value='tfportrear0_0/ll'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/uplink"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
2162025-01-10T21:54:32.457ZINFOSledAgent (ServiceManager): Booting oxz_switch zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_switch
2172025-01-10T21:54:32.698ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 52.032374ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2182025-01-10T21:54:32.902ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 69.720084ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2192025-01-10T21:54:33.102ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 215.434893ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2202025-01-10T21:54:33.135ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 3.371282612s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
2212025-01-10T21:54:33.482ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 207.480462ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2222025-01-10T21:54:33.877ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 598.181586ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2232025-01-10T21:54:34.635ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.40517885s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2242025-01-10T21:54:36.260ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 906.993505ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2252025-01-10T21:54:36.508ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 11.599368012s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
2262025-01-10T21:54:37.212ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 932.641548ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2272025-01-10T21:54:38.532ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/2235d715-657e-4a92-8f59-9a6c0c957b73) file = sled-agent/src/nexus.rs:527 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
2282025-01-10T21:54:38.562ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
2292025-01-10T21:54:38.562ZINFOSledAgent (dropshot (Repo Depot)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:12348 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
2302025-01-10T21:54:38.564ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:12345
2312025-01-10T21:54:38.564ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:432
2322025-01-10T21:54:38.564ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
2332025-01-10T21:54:38.564ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2342025-01-10T21:54:38.564ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/10481327-13d1-4548-950f-130d063e2690/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2352025-01-10T21:54:38.564ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:138
2362025-01-10T21:54:38.564ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56 }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [PortConfigV2 { routes: [RouteConfig { destination: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), nexthop: 10.151.2.100, vlan_id: None, rib_priority: None }], addresses: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: Some(None), bgp_peers: [], autoneg: false, lldp: None, tx_eq: None }], bgp: [], bfd: [] })))
2372025-01-10T21:54:38.564ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3973
2382025-01-10T21:54:38.564ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:81 remote_addr = [fdb0:18c0:4d0d:a02a::1]:37010
2392025-01-10T21:54:38.777ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:117 peer = [fdb0:18c0:4d0d:a02a::1]:12346
2402025-01-10T21:54:38.777ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:252 target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
2412025-01-10T21:54:38.781ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:4294 new = [fd00:1122:3344:101::2, ::1] old = [::1]
2422025-01-10T21:54:39.775ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/2235d715-657e-4a92-8f59-9a6c0c957b73) file = sled-agent/src/probe_manager.rs:231 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
2432025-01-10T21:54:39.775ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 13884378162807884192 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
2442025-01-10T21:54:39.775ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17833868550910882914 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
2452025-01-10T21:54:39.777ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:63037
2462025-01-10T21:54:39.777ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
2472025-01-10T21:54:39.777ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
2482025-01-10T21:54:39.777ZINFOSledAgent (ServiceManager): No ledger in /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-zones.json file = common/src/ledger.rs:188
2492025-01-10T21:54:39.777ZINFOSledAgent (ServiceManager): No ledger in /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-zones.json file = common/src/ledger.rs:188
2502025-01-10T21:54:39.777ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 253 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:63037 req_id = 6a140716-6691-4b3b-a7d5-7869d2e4077c response_code = 200 uri = /inventory
2512025-01-10T21:54:40.000ZINFOSledAgent (VmmReservoirManager): Finished setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:279 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
2522025-01-10T21:54:40.009ZINFOSledAgent (ServiceManager): refreshed zone-network-setup service with new configuration file = sled-agent/src/services.rs:4343
2532025-01-10T21:54:40.009ZINFOSledAgent (ServiceManager): configuring dendrite service file = sled-agent/src/services.rs:4401
2542025-01-10T21:54:40.039ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:39461
2552025-01-10T21:54:40.039ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:39461 req_id = 8b88a649-068a-4d76-9afb-943ce76b6d26 response_code = 200 uri = /sled-role
2562025-01-10T21:54:40.083ZINFOSledAgent (RSS): attempting to set up sled's storage: OmicronPhysicalDisksConfig { generation: Generation(2), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: ca5e2dc6-2374-42a6-a2d9-672b7c07a1b4 (physical_disk), pool_id: 73828a50-9586-442b-b526-668d3e1de808 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: d0ce559b-2c71-49e1-82c2-faeb2c75c1ab (physical_disk), pool_id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 8f40fd6d-c9a9-4daf-8640-0dee737c8b92 (physical_disk), pool_id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 9fda2a48-f211-4699-8f5a-8f0eddf14734 (physical_disk), pool_id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 3bff7672-5384-4136-9fbd-b3b7f7f49549 (physical_disk), pool_id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 6b74c72f-5558-4188-9849-00581ba99d5c (physical_disk), pool_id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 00137a5a-9493-491b-8596-7832867e087e (physical_disk), pool_id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: 4361bbc0-1335-448f-b658-9cb4f0ac5f51 (physical_disk), pool_id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 9e34ace2-92a3-4270-974f-805889d57d05 (physical_disk), pool_id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool) }] } file = sled-agent/src/rack_setup/service.rs:385 sled_address = [fd00:1122:3344:101::1]:12345
2572025-01-10T21:54:40.083ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:56249
2582025-01-10T21:54:40.083ZINFOSledAgent: physical disks ensure file = sled-agent/src/sled_agent.rs:893 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
2592025-01-10T21:54:40.083ZINFOSledAgent (StorageManager): Received OmicronPhysicalDisksEnsure { config: OmicronPhysicalDisksConfig { generation: Generation(2), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: ca5e2dc6-2374-42a6-a2d9-672b7c07a1b4 (physical_disk), pool_id: 73828a50-9586-442b-b526-668d3e1de808 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: d0ce559b-2c71-49e1-82c2-faeb2c75c1ab (physical_disk), pool_id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 8f40fd6d-c9a9-4daf-8640-0dee737c8b92 (physical_disk), pool_id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 9fda2a48-f211-4699-8f5a-8f0eddf14734 (physical_disk), pool_id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 3bff7672-5384-4136-9fbd-b3b7f7f49549 (physical_disk), pool_id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 6b74c72f-5558-4188-9849-00581ba99d5c (physical_disk), pool_id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 00137a5a-9493-491b-8596-7832867e087e (physical_disk), pool_id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: 4361bbc0-1335-448f-b658-9cb4f0ac5f51 (physical_disk), pool_id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 9e34ace2-92a3-4270-974f-805889d57d05 (physical_disk), pool_id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool) }] }, tx: ... } file = sled-storage/src/manager.rs:607
2602025-01-10T21:54:40.083ZINFOSledAgent (StorageManager): No ledger in /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2612025-01-10T21:54:40.083ZINFOSledAgent (StorageManager): No ledger in /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2622025-01-10T21:54:40.083ZINFOSledAgent (StorageManager): No previously-stored 'requested disks', creating new ledger file = sled-storage/src/manager.rs:1298 request = omicron_physical_disks_ensure
2632025-01-10T21:54:40.083ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
2642025-01-10T21:54:40.083ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:342
2652025-01-10T21:54:40.083ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2662025-01-10T21:54:40.083ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } path = /scratch/u2_0.vdev
2672025-01-10T21:54:40.094ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_0.vdev file = sled-hardware/src/disk.rs:367
2682025-01-10T21:54:40.094ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 73828a50-9586-442b-b526-668d3e1de808 (zpool)
2692025-01-10T21:54:40.193ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }
2702025-01-10T21:54:40.201ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:239
2712025-01-10T21:54:40.201ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:241
2722025-01-10T21:54:40.201ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:245
2732025-01-10T21:54:40.201ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:247
2742025-01-10T21:54:40.201ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2752025-01-10T21:54:40.201ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2762025-01-10T21:54:40.201ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
2772025-01-10T21:54:40.234ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2782025-01-10T21:54:40.242ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_73828a50-9586-442b-b526-668d3e1de808/crypt/zone file = sled-storage/src/dataset.rs:302
2792025-01-10T21:54:40.373ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_73828a50-9586-442b-b526-668d3e1de808 file = sled-storage/src/dataset.rs:388
2802025-01-10T21:54:40.381ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:440
2812025-01-10T21:54:40.381ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:357
2822025-01-10T21:54:40.381ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:342
2832025-01-10T21:54:40.381ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2842025-01-10T21:54:40.381ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } path = /scratch/u2_1.vdev
2852025-01-10T21:54:40.390ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_1.vdev file = sled-hardware/src/disk.rs:367
2862025-01-10T21:54:40.390ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 598e3879-d23d-4354-9e0c-045503f010a6 (zpool)
2872025-01-10T21:54:40.478ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }
2882025-01-10T21:54:40.486ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:239
2892025-01-10T21:54:40.486ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:241
2902025-01-10T21:54:40.486ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:245
2912025-01-10T21:54:40.486ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:247
2922025-01-10T21:54:40.486ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2932025-01-10T21:54:40.486ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2942025-01-10T21:54:40.486ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
2952025-01-10T21:54:40.520ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2962025-01-10T21:54:40.528ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_598e3879-d23d-4354-9e0c-045503f010a6/crypt/zone file = sled-storage/src/dataset.rs:302
2972025-01-10T21:54:40.654ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_598e3879-d23d-4354-9e0c-045503f010a6 file = sled-storage/src/dataset.rs:388
2982025-01-10T21:54:40.662ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:440
2992025-01-10T21:54:40.662ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:357
3002025-01-10T21:54:40.662ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:342
3012025-01-10T21:54:40.662ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3022025-01-10T21:54:40.662ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } path = /scratch/u2_2.vdev
3032025-01-10T21:54:40.671ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_2.vdev file = sled-hardware/src/disk.rs:367
3042025-01-10T21:54:40.671ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool)
3052025-01-10T21:54:40.764ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }
3062025-01-10T21:54:40.772ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:239
3072025-01-10T21:54:40.772ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:241
3082025-01-10T21:54:40.772ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:245
3092025-01-10T21:54:40.772ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:247
3102025-01-10T21:54:40.772ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3112025-01-10T21:54:40.772ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3122025-01-10T21:54:40.772ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
3132025-01-10T21:54:40.777ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/2235d715-657e-4a92-8f59-9a6c0c957b73) file = sled-agent/src/probe_manager.rs:231 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
3142025-01-10T21:54:40.806ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3152025-01-10T21:54:40.813ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_5d7ad0a3-e47e-4369-b599-d02dbd3a8829/crypt/zone file = sled-storage/src/dataset.rs:302
3162025-01-10T21:54:40.944ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_5d7ad0a3-e47e-4369-b599-d02dbd3a8829 file = sled-storage/src/dataset.rs:388
3172025-01-10T21:54:40.952ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:440
3182025-01-10T21:54:40.952ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:357
3192025-01-10T21:54:40.952ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:342
3202025-01-10T21:54:40.952ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3212025-01-10T21:54:40.952ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } path = /scratch/u2_3.vdev
3222025-01-10T21:54:40.962ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_3.vdev file = sled-hardware/src/disk.rs:367
3232025-01-10T21:54:40.962ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = d27ec511-4e48-4e26-9677-295de0452e91 (zpool)
3242025-01-10T21:54:41.058ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }
3252025-01-10T21:54:41.066ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:239
3262025-01-10T21:54:41.066ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:241
3272025-01-10T21:54:41.066ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:245
3282025-01-10T21:54:41.066ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:247
3292025-01-10T21:54:41.066ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3302025-01-10T21:54:41.066ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3312025-01-10T21:54:41.066ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
3322025-01-10T21:54:41.100ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3332025-01-10T21:54:41.108ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_d27ec511-4e48-4e26-9677-295de0452e91/crypt/zone file = sled-storage/src/dataset.rs:302
3342025-01-10T21:54:41.243ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_d27ec511-4e48-4e26-9677-295de0452e91 file = sled-storage/src/dataset.rs:388
3352025-01-10T21:54:41.251ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:440
3362025-01-10T21:54:41.251ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:357
3372025-01-10T21:54:41.251ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:342
3382025-01-10T21:54:41.251ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3392025-01-10T21:54:41.251ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } path = /scratch/u2_4.vdev
3402025-01-10T21:54:41.262ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_4.vdev file = sled-hardware/src/disk.rs:367
3412025-01-10T21:54:41.262ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool)
3422025-01-10T21:54:41.356ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }
3432025-01-10T21:54:41.364ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:239
3442025-01-10T21:54:41.364ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:241
3452025-01-10T21:54:41.364ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:245
3462025-01-10T21:54:41.364ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:247
3472025-01-10T21:54:41.364ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3482025-01-10T21:54:41.364ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3492025-01-10T21:54:41.364ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
3502025-01-10T21:54:41.399ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3512025-01-10T21:54:41.407ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_ce5c629d-252d-4e12-9f9d-2087cb772d81/crypt/zone file = sled-storage/src/dataset.rs:302
3522025-01-10T21:54:41.546ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_ce5c629d-252d-4e12-9f9d-2087cb772d81 file = sled-storage/src/dataset.rs:388
3532025-01-10T21:54:41.555ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:440
3542025-01-10T21:54:41.555ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:357
3552025-01-10T21:54:41.555ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:342
3562025-01-10T21:54:41.555ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3572025-01-10T21:54:41.555ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } path = /scratch/u2_5.vdev
3582025-01-10T21:54:41.566ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_5.vdev file = sled-hardware/src/disk.rs:367
3592025-01-10T21:54:41.566ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool)
3602025-01-10T21:54:41.663ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External }
3612025-01-10T21:54:41.671ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:239
3622025-01-10T21:54:41.671ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:241
3632025-01-10T21:54:41.671ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:245
3642025-01-10T21:54:41.671ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:247
3652025-01-10T21:54:41.671ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3662025-01-10T21:54:41.671ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3672025-01-10T21:54:41.671ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
3682025-01-10T21:54:41.707ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3692025-01-10T21:54:41.714ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_7b0e1b76-af4f-49bd-9848-611d8d8666ad/crypt/zone file = sled-storage/src/dataset.rs:302
3702025-01-10T21:54:41.780ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/2235d715-657e-4a92-8f59-9a6c0c957b73) file = sled-agent/src/probe_manager.rs:231 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
3712025-01-10T21:54:41.843ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_7b0e1b76-af4f-49bd-9848-611d8d8666ad file = sled-storage/src/dataset.rs:388
3722025-01-10T21:54:41.850ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:440
3732025-01-10T21:54:41.850ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:357
3742025-01-10T21:54:41.850ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:342
3752025-01-10T21:54:41.850ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3762025-01-10T21:54:41.850ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } path = /scratch/u2_6.vdev
3772025-01-10T21:54:41.860ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_6.vdev file = sled-hardware/src/disk.rs:367
3782025-01-10T21:54:41.860ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 619e5037-b196-43f6-b092-666b1b1387ff (zpool)
3792025-01-10T21:54:41.952ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }
3802025-01-10T21:54:41.960ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:239
3812025-01-10T21:54:41.960ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:241
3822025-01-10T21:54:41.960ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:245
3832025-01-10T21:54:41.960ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:247
3842025-01-10T21:54:41.960ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3852025-01-10T21:54:41.960ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3862025-01-10T21:54:41.960ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
3872025-01-10T21:54:41.995ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3882025-01-10T21:54:42.003ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_619e5037-b196-43f6-b092-666b1b1387ff/crypt/zone file = sled-storage/src/dataset.rs:302
3892025-01-10T21:54:42.138ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_619e5037-b196-43f6-b092-666b1b1387ff file = sled-storage/src/dataset.rs:388
3902025-01-10T21:54:42.148ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:440
3912025-01-10T21:54:42.148ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:357
3922025-01-10T21:54:42.148ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:342
3932025-01-10T21:54:42.148ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3942025-01-10T21:54:42.148ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } path = /scratch/u2_7.vdev
3952025-01-10T21:54:42.153ZINFOSledAgent (ServiceManager): refreshed dendrite service with new configuration file = sled-agent/src/services.rs:4467
3962025-01-10T21:54:42.153ZINFOSledAgent (ServiceManager): configuring lldp service file = sled-agent/src/services.rs:4494
3972025-01-10T21:54:42.159ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_7.vdev file = sled-hardware/src/disk.rs:367
3982025-01-10T21:54:42.160ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = eecab578-d379-46e4-90f5-9dc11839dad5 (zpool)
3992025-01-10T21:54:42.256ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }
4002025-01-10T21:54:42.264ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:239
4012025-01-10T21:54:42.264ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:241
4022025-01-10T21:54:42.264ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:245
4032025-01-10T21:54:42.264ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:247
4042025-01-10T21:54:42.264ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4052025-01-10T21:54:42.264ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4062025-01-10T21:54:42.264ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
4072025-01-10T21:54:42.300ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4082025-01-10T21:54:42.308ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_eecab578-d379-46e4-90f5-9dc11839dad5/crypt/zone file = sled-storage/src/dataset.rs:302
4092025-01-10T21:54:42.445ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_eecab578-d379-46e4-90f5-9dc11839dad5 file = sled-storage/src/dataset.rs:388
4102025-01-10T21:54:42.452ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:440
4112025-01-10T21:54:42.452ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:357
4122025-01-10T21:54:42.452ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:342
4132025-01-10T21:54:42.452ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4142025-01-10T21:54:42.452ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } path = /scratch/u2_8.vdev
4152025-01-10T21:54:42.463ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_8.vdev file = sled-hardware/src/disk.rs:367
4162025-01-10T21:54:42.463ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool)
4172025-01-10T21:54:42.552ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }
4182025-01-10T21:54:42.559ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:239
4192025-01-10T21:54:42.559ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:241
4202025-01-10T21:54:42.559ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:245
4212025-01-10T21:54:42.559ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:247
4222025-01-10T21:54:42.559ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4232025-01-10T21:54:42.559ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4242025-01-10T21:54:42.559ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
4252025-01-10T21:54:42.590ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4262025-01-10T21:54:42.597ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_784c4a6b-3804-44fa-afd4-c974bef9dd5d/crypt/zone file = sled-storage/src/dataset.rs:302
4272025-01-10T21:54:42.740ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_784c4a6b-3804-44fa-afd4-c974bef9dd5d file = sled-storage/src/dataset.rs:388
4282025-01-10T21:54:42.747ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:440
4292025-01-10T21:54:42.747ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:357
4302025-01-10T21:54:42.747ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4312025-01-10T21:54:42.747ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(2), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 53b46f94-5cb9-4bbb-a551-b5457accb3ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 10481327-13d1-4548-950f-130d063e2690 (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
4322025-01-10T21:54:42.747ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
4332025-01-10T21:54:42.747ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
4342025-01-10T21:54:42.747ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/10481327-13d1-4548-950f-130d063e2690/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4352025-01-10T21:54:42.748ZINFOSledAgent: physical disks ensure: Updated storage file = sled-agent/src/sled_agent.rs:897 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
4362025-01-10T21:54:42.748ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
4372025-01-10T21:54:42.748ZINFOSledAgent: physical disks ensure: Propagating new generation of disks file = sled-agent/src/sled_agent.rs:923 generation = Generation(2) sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
4382025-01-10T21:54:42.748ZINFOSledAgent: physical disks ensure: Updated storage monitor file = sled-agent/src/sled_agent.rs:928 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
4392025-01-10T21:54:42.748ZINFOSledAgent: physical disks ensure: Updated zone bundler file = sled-agent/src/sled_agent.rs:933 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
4402025-01-10T21:54:42.748ZINFOSledAgent (ProbeManager): use_only_these_disks: Processing new request file = sled-agent/src/probe_manager.rs:122 gen = Generation(2) sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
4412025-01-10T21:54:42.748ZINFOSledAgent: physical disks ensure: Updated probes file = sled-agent/src/sled_agent.rs:939 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
4422025-01-10T21:54:42.748ZINFOSledAgent (InstanceManager): use_only_these_disks: Processing new request file = sled-agent/src/instance_manager.rs:799 gen = Generation(2)
4432025-01-10T21:54:42.748ZINFOSledAgent: physical disks ensure: Updated instances file = sled-agent/src/sled_agent.rs:944 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
4442025-01-10T21:54:42.748ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2664738 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:56249 req_id = 547aa931-4403-405d-a2a9-86c7b4b1a773 response_code = 200 uri = /omicron-physical-disks
4452025-01-10T21:54:42.780ZINFOSledAgent (RSS): attempting to set up sled's datasets: DatasetsConfig { generation: Generation(1), datasets: {008e98df-342b-41c5-a783-b78a34730787 (dataset): DatasetConfig { id: 008e98df-342b-41c5-a783-b78a34730787 (dataset), name: DatasetName { pool_name: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 03203c1e-8ef2-4734-9099-c20371a47ece (dataset): DatasetConfig { id: 03203c1e-8ef2-4734-9099-c20371a47ece (dataset), name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 063c71c5-3b5e-469d-89af-eb0fb33a17fd (dataset): DatasetConfig { id: 063c71c5-3b5e-469d-89af-eb0fb33a17fd (dataset), name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 09f632b4-c3ad-46e7-ada0-e0ec568c0eca (dataset): DatasetConfig { id: 09f632b4-c3ad-46e7-ada0-e0ec568c0eca (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 0ea1c1bd-6a80-43c0-b7e3-1d4ef5e6ef61 (dataset): DatasetConfig { id: 0ea1c1bd-6a80-43c0-b7e3-1d4ef5e6ef61 (dataset), name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 1b66b8db-c4ca-4682-89d0-de08045b80e0 (dataset): DatasetConfig { id: 1b66b8db-c4ca-4682-89d0-de08045b80e0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1d8cde5b-73a6-4157-badc-abc4ff4d225e (dataset): DatasetConfig { id: 1d8cde5b-73a6-4157-badc-abc4ff4d225e (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: TransientZone { name: "oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 21f707ac-bf55-41c7-94fe-00bda31e6282 (dataset): DatasetConfig { id: 21f707ac-bf55-41c7-94fe-00bda31e6282 (dataset), name: DatasetName { pool_name: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, kind: TransientZone { name: "oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 22847bf5-bc80-404a-b8e1-ea6b79d8808f (dataset): DatasetConfig { id: 22847bf5-bc80-404a-b8e1-ea6b79d8808f (dataset), name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 24d6b063-e5bd-40d8-b6d5-e2786a7b18c3 (dataset): DatasetConfig { id: 24d6b063-e5bd-40d8-b6d5-e2786a7b18c3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2503703a-4148-421b-8005-d41a9833af5e (dataset): DatasetConfig { id: 2503703a-4148-421b-8005-d41a9833af5e (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 29a5960c-f482-48b3-8e24-44b4980067ad (dataset): DatasetConfig { id: 29a5960c-f482-48b3-8e24-44b4980067ad (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2ed25107-4038-4f93-8958-1dd59e1dcb88 (dataset): DatasetConfig { id: 2ed25107-4038-4f93-8958-1dd59e1dcb88 (dataset), name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 33c827cd-057a-4d4d-ba81-1a5f329e4a89 (dataset): DatasetConfig { id: 33c827cd-057a-4d4d-ba81-1a5f329e4a89 (dataset), name: DatasetName { pool_name: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 40d5a28f-bd02-46e6-a714-f9c9c27aeb67 (dataset): DatasetConfig { id: 40d5a28f-bd02-46e6-a714-f9c9c27aeb67 (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 438123c8-bb30-4953-a6f9-dbc284442812 (dataset): DatasetConfig { id: 438123c8-bb30-4953-a6f9-dbc284442812 (dataset), name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 45c99f83-d166-4749-852b-78a851555991 (dataset): DatasetConfig { id: 45c99f83-d166-4749-852b-78a851555991 (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 464fb0c4-51fb-409c-ac51-98457c3aeaaf (dataset): DatasetConfig { id: 464fb0c4-51fb-409c-ac51-98457c3aeaaf (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 47f1e3e3-ab1c-4d07-ab22-0d5a58cf3131 (dataset): DatasetConfig { id: 47f1e3e3-ab1c-4d07-ab22-0d5a58cf3131 (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4da7b152-1745-4d43-8dd9-51173ab5663c (dataset): DatasetConfig { id: 4da7b152-1745-4d43-8dd9-51173ab5663c (dataset), name: DatasetName { pool_name: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 59ef33c1-79f2-4339-8f04-f3c9c93f7a17 (dataset): DatasetConfig { id: 59ef33c1-79f2-4339-8f04-f3c9c93f7a17 (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6043f36a-beb0-49ff-a217-8277edca9311 (dataset): DatasetConfig { id: 6043f36a-beb0-49ff-a217-8277edca9311 (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 628c4e06-780c-4b9a-aa6e-57459f68c4b4 (dataset): DatasetConfig { id: 628c4e06-780c-4b9a-aa6e-57459f68c4b4 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6687339d-1e73-408c-bbfa-6a8ef7f840fe (dataset): DatasetConfig { id: 6687339d-1e73-408c-bbfa-6a8ef7f840fe (dataset), name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6c3d77f1-1148-439a-b2f1-a6fd3e52be16 (dataset): DatasetConfig { id: 6c3d77f1-1148-439a-b2f1-a6fd3e52be16 (dataset), name: DatasetName { pool_name: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 70a6fb89-4a60-49f7-8915-475c1e78601a (dataset): DatasetConfig { id: 70a6fb89-4a60-49f7-8915-475c1e78601a (dataset), name: DatasetName { pool_name: ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 733cfa76-504d-499e-b087-e1d475de9bb8 (dataset): DatasetConfig { id: 733cfa76-504d-499e-b087-e1d475de9bb8 (dataset), name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 750c9cbd-c10c-44da-9f72-ccfa6d609913 (dataset): DatasetConfig { id: 750c9cbd-c10c-44da-9f72-ccfa6d609913 (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 75ffc865-c85b-4ec8-bebc-a948259b6724 (dataset): DatasetConfig { id: 75ffc865-c85b-4ec8-bebc-a948259b6724 (dataset), name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 80d7e612-15d1-4aa5-8717-d4e71c5aa7fa (dataset): DatasetConfig { id: 80d7e612-15d1-4aa5-8717-d4e71c5aa7fa (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Clickhouse }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 83185423-3ebe-4908-bde3-8e690f699651 (dataset): DatasetConfig { id: 83185423-3ebe-4908-bde3-8e690f699651 (dataset), name: DatasetName { pool_name: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 85d42e9d-6712-456b-adda-24c3931acd6a (dataset): DatasetConfig { id: 85d42e9d-6712-456b-adda-24c3931acd6a (dataset), name: DatasetName { pool_name: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 86ce31a1-3bea-4ac3-aca5-20b1d0cc3fbb (dataset): DatasetConfig { id: 86ce31a1-3bea-4ac3-aca5-20b1d0cc3fbb (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 91bb46d8-b239-40bb-a00f-d5bfb584fe1d (dataset): DatasetConfig { id: 91bb46d8-b239-40bb-a00f-d5bfb584fe1d (dataset), name: DatasetName { pool_name: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 91edcf22-ce9d-49ac-943e-49ecef57e579 (dataset): DatasetConfig { id: 91edcf22-ce9d-49ac-943e-49ecef57e579 (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_bce93894-738c-4596-b983-281da931e977" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 92e54b01-8408-4cda-9a85-b931d664b066 (dataset): DatasetConfig { id: 92e54b01-8408-4cda-9a85-b931d664b066 (dataset), name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 93d79b41-1149-43fa-b7f4-c78163c8ded1 (dataset): DatasetConfig { id: 93d79b41-1149-43fa-b7f4-c78163c8ded1 (dataset), name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9b5b645d-69e4-4703-9eb3-626fe9e0d67f (dataset): DatasetConfig { id: 9b5b645d-69e4-4703-9eb3-626fe9e0d67f (dataset), name: DatasetName { pool_name: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9c524f5c-2a99-4d9c-bd76-8f88fea7c716 (dataset): DatasetConfig { id: 9c524f5c-2a99-4d9c-bd76-8f88fea7c716 (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 9ce0b36f-810b-425b-b55f-6f7aa6c6c710 (dataset): DatasetConfig { id: 9ce0b36f-810b-425b-b55f-6f7aa6c6c710 (dataset), name: DatasetName { pool_name: ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, a2c3fddf-20ea-418b-852b-52ba54568248 (dataset): DatasetConfig { id: a2c3fddf-20ea-418b-852b-52ba54568248 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ab069e64-cfcd-47f0-8587-551b28cf78a6 (dataset): DatasetConfig { id: ab069e64-cfcd-47f0-8587-551b28cf78a6 (dataset), name: DatasetName { pool_name: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }, kind: TransientZone { name: "oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ab771488-33e3-4be3-b26c-114c437ab976 (dataset): DatasetConfig { id: ab771488-33e3-4be3-b26c-114c437ab976 (dataset), name: DatasetName { pool_name: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, ac34b7dc-0ed3-4817-b7bd-6fdbc2125c09 (dataset): DatasetConfig { id: ac34b7dc-0ed3-4817-b7bd-6fdbc2125c09 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, aea7738e-cc37-478f-ab23-1727f0d35848 (dataset): DatasetConfig { id: aea7738e-cc37-478f-ab23-1727f0d35848 (dataset), name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b2677897-ec61-4901-aad0-7cf4fb5a11e3 (dataset): DatasetConfig { id: b2677897-ec61-4901-aad0-7cf4fb5a11e3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b48b8c31-968e-4d77-8beb-44213e4ee26a (dataset): DatasetConfig { id: b48b8c31-968e-4d77-8beb-44213e4ee26a (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b9f7455d-da77-425d-9aea-b7db15e2a2a3 (dataset): DatasetConfig { id: b9f7455d-da77-425d-9aea-b7db15e2a2a3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, bda762ab-d36c-4eb6-b234-39989954e4f0 (dataset): DatasetConfig { id: bda762ab-d36c-4eb6-b234-39989954e4f0 (dataset), name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c07f9afc-884c-42a9-9246-3be0054f6ea0 (dataset): DatasetConfig { id: c07f9afc-884c-42a9-9246-3be0054f6ea0 (dataset), name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, c210ec8f-f09e-4816-a31d-8c801f0dca3a (dataset): DatasetConfig { id: c210ec8f-f09e-4816-a31d-8c801f0dca3a (dataset), name: DatasetName { pool_name: ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ca9bcc52-c62b-4aff-a65c-9ba14b5debe1 (dataset): DatasetConfig { id: ca9bcc52-c62b-4aff-a65c-9ba14b5debe1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ceaac012-6b30-4180-b9bf-0b2a93462704 (dataset): DatasetConfig { id: ceaac012-6b30-4180-b9bf-0b2a93462704 (dataset), name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d1849f3b-328c-4cc2-8413-4d270bb68600 (dataset): DatasetConfig { id: d1849f3b-328c-4cc2-8413-4d270bb68600 (dataset), name: DatasetName { pool_name: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d2442e44-302d-4407-a87e-4ea3af0ea38c (dataset): DatasetConfig { id: d2442e44-302d-4407-a87e-4ea3af0ea38c (dataset), name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d34e4a43-8517-4a5d-9790-1a475b2394fb (dataset): DatasetConfig { id: d34e4a43-8517-4a5d-9790-1a475b2394fb (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d7da17ee-f6a0-481e-abbd-6c8fb9354dcf (dataset): DatasetConfig { id: d7da17ee-f6a0-481e-abbd-6c8fb9354dcf (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d8832384-f4b6-43be-859f-4202abaa3e19 (dataset): DatasetConfig { id: d8832384-f4b6-43be-859f-4202abaa3e19 (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d8f0d44a-0729-422d-a791-0fc93210cd57 (dataset): DatasetConfig { id: d8f0d44a-0729-422d-a791-0fc93210cd57 (dataset), name: DatasetName { pool_name: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e2519026-d893-4d7c-9d4d-0327f43ee8ec (dataset): DatasetConfig { id: e2519026-d893-4d7c-9d4d-0327f43ee8ec (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e99452a3-0604-4667-8ed0-159c9188f80a (dataset): DatasetConfig { id: e99452a3-0604-4667-8ed0-159c9188f80a (dataset), name: DatasetName { pool_name: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ecafbc56-6a0d-42d0-a8cb-27b21796beaa (dataset): DatasetConfig { id: ecafbc56-6a0d-42d0-a8cb-27b21796beaa (dataset), name: DatasetName { pool_name: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, ed298c88-2fa5-4c50-b63b-aa9569ac23a4 (dataset): DatasetConfig { id: ed298c88-2fa5-4c50-b63b-aa9569ac23a4 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f0f48c4f-3882-4289-ad9c-405972710c4e (dataset): DatasetConfig { id: f0f48c4f-3882-4289-ad9c-405972710c4e (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f494c03a-667c-4e4d-9e76-ad6404a19706 (dataset): DatasetConfig { id: f494c03a-667c-4e4d-9e76-ad6404a19706 (dataset), name: DatasetName { pool_name: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f6ddc144-ffcd-49cf-9d53-ed8ca9b538bd (dataset): DatasetConfig { id: f6ddc144-ffcd-49cf-9d53-ed8ca9b538bd (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }} } file = sled-agent/src/rack_setup/service.rs:460 sled_address = [fd00:1122:3344:101::1]:12345
4462025-01-10T21:54:42.780ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:35668
4472025-01-10T21:54:42.780ZINFOSledAgent: datasets ensure file = sled-agent/src/sled_agent.rs:859 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
4482025-01-10T21:54:42.780ZINFOSledAgent (StorageManager): Received DatasetsEnsure { config: DatasetsConfig { generation: Generation(1), datasets: {008e98df-342b-41c5-a783-b78a34730787 (dataset): DatasetConfig { id: 008e98df-342b-41c5-a783-b78a34730787 (dataset), name: DatasetName { pool_name: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 03203c1e-8ef2-4734-9099-c20371a47ece (dataset): DatasetConfig { id: 03203c1e-8ef2-4734-9099-c20371a47ece (dataset), name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 063c71c5-3b5e-469d-89af-eb0fb33a17fd (dataset): DatasetConfig { id: 063c71c5-3b5e-469d-89af-eb0fb33a17fd (dataset), name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 09f632b4-c3ad-46e7-ada0-e0ec568c0eca (dataset): DatasetConfig { id: 09f632b4-c3ad-46e7-ada0-e0ec568c0eca (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 0ea1c1bd-6a80-43c0-b7e3-1d4ef5e6ef61 (dataset): DatasetConfig { id: 0ea1c1bd-6a80-43c0-b7e3-1d4ef5e6ef61 (dataset), name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 1b66b8db-c4ca-4682-89d0-de08045b80e0 (dataset): DatasetConfig { id: 1b66b8db-c4ca-4682-89d0-de08045b80e0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1d8cde5b-73a6-4157-badc-abc4ff4d225e (dataset): DatasetConfig { id: 1d8cde5b-73a6-4157-badc-abc4ff4d225e (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: TransientZone { name: "oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 21f707ac-bf55-41c7-94fe-00bda31e6282 (dataset): DatasetConfig { id: 21f707ac-bf55-41c7-94fe-00bda31e6282 (dataset), name: DatasetName { pool_name: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, kind: TransientZone { name: "oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 22847bf5-bc80-404a-b8e1-ea6b79d8808f (dataset): DatasetConfig { id: 22847bf5-bc80-404a-b8e1-ea6b79d8808f (dataset), name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 24d6b063-e5bd-40d8-b6d5-e2786a7b18c3 (dataset): DatasetConfig { id: 24d6b063-e5bd-40d8-b6d5-e2786a7b18c3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2503703a-4148-421b-8005-d41a9833af5e (dataset): DatasetConfig { id: 2503703a-4148-421b-8005-d41a9833af5e (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 29a5960c-f482-48b3-8e24-44b4980067ad (dataset): DatasetConfig { id: 29a5960c-f482-48b3-8e24-44b4980067ad (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2ed25107-4038-4f93-8958-1dd59e1dcb88 (dataset): DatasetConfig { id: 2ed25107-4038-4f93-8958-1dd59e1dcb88 (dataset), name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 33c827cd-057a-4d4d-ba81-1a5f329e4a89 (dataset): DatasetConfig { id: 33c827cd-057a-4d4d-ba81-1a5f329e4a89 (dataset), name: DatasetName { pool_name: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 40d5a28f-bd02-46e6-a714-f9c9c27aeb67 (dataset): DatasetConfig { id: 40d5a28f-bd02-46e6-a714-f9c9c27aeb67 (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 438123c8-bb30-4953-a6f9-dbc284442812 (dataset): DatasetConfig { id: 438123c8-bb30-4953-a6f9-dbc284442812 (dataset), name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 45c99f83-d166-4749-852b-78a851555991 (dataset): DatasetConfig { id: 45c99f83-d166-4749-852b-78a851555991 (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 464fb0c4-51fb-409c-ac51-98457c3aeaaf (dataset): DatasetConfig { id: 464fb0c4-51fb-409c-ac51-98457c3aeaaf (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 47f1e3e3-ab1c-4d07-ab22-0d5a58cf3131 (dataset): DatasetConfig { id: 47f1e3e3-ab1c-4d07-ab22-0d5a58cf3131 (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4da7b152-1745-4d43-8dd9-51173ab5663c (dataset): DatasetConfig { id: 4da7b152-1745-4d43-8dd9-51173ab5663c (dataset), name: DatasetName { pool_name: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 59ef33c1-79f2-4339-8f04-f3c9c93f7a17 (dataset): DatasetConfig { id: 59ef33c1-79f2-4339-8f04-f3c9c93f7a17 (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6043f36a-beb0-49ff-a217-8277edca9311 (dataset): DatasetConfig { id: 6043f36a-beb0-49ff-a217-8277edca9311 (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 628c4e06-780c-4b9a-aa6e-57459f68c4b4 (dataset): DatasetConfig { id: 628c4e06-780c-4b9a-aa6e-57459f68c4b4 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6687339d-1e73-408c-bbfa-6a8ef7f840fe (dataset): DatasetConfig { id: 6687339d-1e73-408c-bbfa-6a8ef7f840fe (dataset), name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6c3d77f1-1148-439a-b2f1-a6fd3e52be16 (dataset): DatasetConfig { id: 6c3d77f1-1148-439a-b2f1-a6fd3e52be16 (dataset), name: DatasetName { pool_name: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 70a6fb89-4a60-49f7-8915-475c1e78601a (dataset): DatasetConfig { id: 70a6fb89-4a60-49f7-8915-475c1e78601a (dataset), name: DatasetName { pool_name: ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 733cfa76-504d-499e-b087-e1d475de9bb8 (dataset): DatasetConfig { id: 733cfa76-504d-499e-b087-e1d475de9bb8 (dataset), name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 750c9cbd-c10c-44da-9f72-ccfa6d609913 (dataset): DatasetConfig { id: 750c9cbd-c10c-44da-9f72-ccfa6d609913 (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 75ffc865-c85b-4ec8-bebc-a948259b6724 (dataset): DatasetConfig { id: 75ffc865-c85b-4ec8-bebc-a948259b6724 (dataset), name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 80d7e612-15d1-4aa5-8717-d4e71c5aa7fa (dataset): DatasetConfig { id: 80d7e612-15d1-4aa5-8717-d4e71c5aa7fa (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Clickhouse }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 83185423-3ebe-4908-bde3-8e690f699651 (dataset): DatasetConfig { id: 83185423-3ebe-4908-bde3-8e690f699651 (dataset), name: DatasetName { pool_name: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 85d42e9d-6712-456b-adda-24c3931acd6a (dataset): DatasetConfig { id: 85d42e9d-6712-456b-adda-24c3931acd6a (dataset), name: DatasetName { pool_name: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 86ce31a1-3bea-4ac3-aca5-20b1d0cc3fbb (dataset): DatasetConfig { id: 86ce31a1-3bea-4ac3-aca5-20b1d0cc3fbb (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 91bb46d8-b239-40bb-a00f-d5bfb584fe1d (dataset): DatasetConfig { id: 91bb46d8-b239-40bb-a00f-d5bfb584fe1d (dataset), name: DatasetName { pool_name: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 91edcf22-ce9d-49ac-943e-49ecef57e579 (dataset): DatasetConfig { id: 91edcf22-ce9d-49ac-943e-49ecef57e579 (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_bce93894-738c-4596-b983-281da931e977" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 92e54b01-8408-4cda-9a85-b931d664b066 (dataset): DatasetConfig { id: 92e54b01-8408-4cda-9a85-b931d664b066 (dataset), name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 93d79b41-1149-43fa-b7f4-c78163c8ded1 (dataset): DatasetConfig { id: 93d79b41-1149-43fa-b7f4-c78163c8ded1 (dataset), name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9b5b645d-69e4-4703-9eb3-626fe9e0d67f (dataset): DatasetConfig { id: 9b5b645d-69e4-4703-9eb3-626fe9e0d67f (dataset), name: DatasetName { pool_name: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9c524f5c-2a99-4d9c-bd76-8f88fea7c716 (dataset): DatasetConfig { id: 9c524f5c-2a99-4d9c-bd76-8f88fea7c716 (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 9ce0b36f-810b-425b-b55f-6f7aa6c6c710 (dataset): DatasetConfig { id: 9ce0b36f-810b-425b-b55f-6f7aa6c6c710 (dataset), name: DatasetName { pool_name: ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, a2c3fddf-20ea-418b-852b-52ba54568248 (dataset): DatasetConfig { id: a2c3fddf-20ea-418b-852b-52ba54568248 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ab069e64-cfcd-47f0-8587-551b28cf78a6 (dataset): DatasetConfig { id: ab069e64-cfcd-47f0-8587-551b28cf78a6 (dataset), name: DatasetName { pool_name: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }, kind: TransientZone { name: "oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ab771488-33e3-4be3-b26c-114c437ab976 (dataset): DatasetConfig { id: ab771488-33e3-4be3-b26c-114c437ab976 (dataset), name: DatasetName { pool_name: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, ac34b7dc-0ed3-4817-b7bd-6fdbc2125c09 (dataset): DatasetConfig { id: ac34b7dc-0ed3-4817-b7bd-6fdbc2125c09 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, aea7738e-cc37-478f-ab23-1727f0d35848 (dataset): DatasetConfig { id: aea7738e-cc37-478f-ab23-1727f0d35848 (dataset), name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b2677897-ec61-4901-aad0-7cf4fb5a11e3 (dataset): DatasetConfig { id: b2677897-ec61-4901-aad0-7cf4fb5a11e3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b48b8c31-968e-4d77-8beb-44213e4ee26a (dataset): DatasetConfig { id: b48b8c31-968e-4d77-8beb-44213e4ee26a (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b9f7455d-da77-425d-9aea-b7db15e2a2a3 (dataset): DatasetConfig { id: b9f7455d-da77-425d-9aea-b7db15e2a2a3 (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, bda762ab-d36c-4eb6-b234-39989954e4f0 (dataset): DatasetConfig { id: bda762ab-d36c-4eb6-b234-39989954e4f0 (dataset), name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c07f9afc-884c-42a9-9246-3be0054f6ea0 (dataset): DatasetConfig { id: c07f9afc-884c-42a9-9246-3be0054f6ea0 (dataset), name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, c210ec8f-f09e-4816-a31d-8c801f0dca3a (dataset): DatasetConfig { id: c210ec8f-f09e-4816-a31d-8c801f0dca3a (dataset), name: DatasetName { pool_name: ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ca9bcc52-c62b-4aff-a65c-9ba14b5debe1 (dataset): DatasetConfig { id: ca9bcc52-c62b-4aff-a65c-9ba14b5debe1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ceaac012-6b30-4180-b9bf-0b2a93462704 (dataset): DatasetConfig { id: ceaac012-6b30-4180-b9bf-0b2a93462704 (dataset), name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d1849f3b-328c-4cc2-8413-4d270bb68600 (dataset): DatasetConfig { id: d1849f3b-328c-4cc2-8413-4d270bb68600 (dataset), name: DatasetName { pool_name: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d2442e44-302d-4407-a87e-4ea3af0ea38c (dataset): DatasetConfig { id: d2442e44-302d-4407-a87e-4ea3af0ea38c (dataset), name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d34e4a43-8517-4a5d-9790-1a475b2394fb (dataset): DatasetConfig { id: d34e4a43-8517-4a5d-9790-1a475b2394fb (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d7da17ee-f6a0-481e-abbd-6c8fb9354dcf (dataset): DatasetConfig { id: d7da17ee-f6a0-481e-abbd-6c8fb9354dcf (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d8832384-f4b6-43be-859f-4202abaa3e19 (dataset): DatasetConfig { id: d8832384-f4b6-43be-859f-4202abaa3e19 (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d8f0d44a-0729-422d-a791-0fc93210cd57 (dataset): DatasetConfig { id: d8f0d44a-0729-422d-a791-0fc93210cd57 (dataset), name: DatasetName { pool_name: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e2519026-d893-4d7c-9d4d-0327f43ee8ec (dataset): DatasetConfig { id: e2519026-d893-4d7c-9d4d-0327f43ee8ec (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e99452a3-0604-4667-8ed0-159c9188f80a (dataset): DatasetConfig { id: e99452a3-0604-4667-8ed0-159c9188f80a (dataset), name: DatasetName { pool_name: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ecafbc56-6a0d-42d0-a8cb-27b21796beaa (dataset): DatasetConfig { id: ecafbc56-6a0d-42d0-a8cb-27b21796beaa (dataset), name: DatasetName { pool_name: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, ed298c88-2fa5-4c50-b63b-aa9569ac23a4 (dataset): DatasetConfig { id: ed298c88-2fa5-4c50-b63b-aa9569ac23a4 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f0f48c4f-3882-4289-ad9c-405972710c4e (dataset): DatasetConfig { id: f0f48c4f-3882-4289-ad9c-405972710c4e (dataset), name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f494c03a-667c-4e4d-9e76-ad6404a19706 (dataset): DatasetConfig { id: f494c03a-667c-4e4d-9e76-ad6404a19706 (dataset), name: DatasetName { pool_name: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f6ddc144-ffcd-49cf-9d53-ed8ca9b538bd (dataset): DatasetConfig { id: f6ddc144-ffcd-49cf-9d53-ed8ca9b538bd (dataset), name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }} }, tx: ... } file = sled-storage/src/manager.rs:607
4492025-01-10T21:54:42.780ZINFOSledAgent (StorageManager): No ledger in /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-datasets.json file = common/src/ledger.rs:188 request = datasets_ensure
4502025-01-10T21:54:42.780ZINFOSledAgent (StorageManager): No ledger in /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-datasets.json file = common/src/ledger.rs:188 request = datasets_ensure
4512025-01-10T21:54:42.780ZINFOSledAgent (StorageManager): No previously-stored 'requested datasets', creating new ledger file = sled-storage/src/manager.rs:909 request = datasets_ensure
4522025-01-10T21:54:42.785ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/2235d715-657e-4a92-8f59-9a6c0c957b73) file = sled-agent/src/probe_manager.rs:231 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
4532025-01-10T21:54:42.796ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4542025-01-10T21:54:42.796ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
4552025-01-10T21:54:42.796ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/784c4a6b-3804-44fa-afd4-c974bef9dd5d/crypt/debug"), full_name: "oxp_784c4a6b-3804-44fa-afd4-c974bef9dd5d/crypt/debug" } file = sled-storage/src/manager.rs:1431
4562025-01-10T21:54:42.825ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4572025-01-10T21:54:42.825ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
4582025-01-10T21:54:42.825ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/5d7ad0a3-e47e-4369-b599-d02dbd3a8829/crypt/zone"), full_name: "oxp_5d7ad0a3-e47e-4369-b599-d02dbd3a8829/crypt/zone" } file = sled-storage/src/manager.rs:1431
4592025-01-10T21:54:42.853ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4602025-01-10T21:54:42.853ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4612025-01-10T21:54:42.853ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/d27ec511-4e48-4e26-9677-295de0452e91/crypt/zone/oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2"), full_name: "oxp_d27ec511-4e48-4e26-9677-295de0452e91/crypt/zone/oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2" } file = sled-storage/src/manager.rs:1431
4622025-01-10T21:54:42.912ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4632025-01-10T21:54:42.912ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
4642025-01-10T21:54:42.912ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/73828a50-9586-442b-b526-668d3e1de808/crypt/debug"), full_name: "oxp_73828a50-9586-442b-b526-668d3e1de808/crypt/debug" } file = sled-storage/src/manager.rs:1431
4652025-01-10T21:54:42.940ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4662025-01-10T21:54:42.941ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
4672025-01-10T21:54:42.941ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/ce5c629d-252d-4e12-9f9d-2087cb772d81/crypt/debug"), full_name: "oxp_ce5c629d-252d-4e12-9f9d-2087cb772d81/crypt/debug" } file = sled-storage/src/manager.rs:1431
4682025-01-10T21:54:42.968ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4692025-01-10T21:54:42.968ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4702025-01-10T21:54:42.968ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_598e3879-d23d-4354-9e0c-045503f010a6/crypt/external_dns" } file = sled-storage/src/manager.rs:1431
4712025-01-10T21:54:43.019ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4722025-01-10T21:54:43.019ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4732025-01-10T21:54:43.019ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/73828a50-9586-442b-b526-668d3e1de808/crypt/zone/oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee"), full_name: "oxp_73828a50-9586-442b-b526-668d3e1de808/crypt/zone/oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee" } file = sled-storage/src/manager.rs:1431
4742025-01-10T21:54:43.078ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4752025-01-10T21:54:43.078ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4762025-01-10T21:54:43.078ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/784c4a6b-3804-44fa-afd4-c974bef9dd5d/crypt/zone/oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603"), full_name: "oxp_784c4a6b-3804-44fa-afd4-c974bef9dd5d/crypt/zone/oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603" } file = sled-storage/src/manager.rs:1431
4772025-01-10T21:54:43.137ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4782025-01-10T21:54:43.137ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
4792025-01-10T21:54:43.137ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/5d7ad0a3-e47e-4369-b599-d02dbd3a8829/crypt/debug"), full_name: "oxp_5d7ad0a3-e47e-4369-b599-d02dbd3a8829/crypt/debug" } file = sled-storage/src/manager.rs:1431
4802025-01-10T21:54:43.165ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4812025-01-10T21:54:43.165ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4822025-01-10T21:54:43.165ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_7b0e1b76-af4f-49bd-9848-611d8d8666ad/crucible" } file = sled-storage/src/manager.rs:1431
4832025-01-10T21:54:43.216ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4842025-01-10T21:54:43.216ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4852025-01-10T21:54:43.216ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_598e3879-d23d-4354-9e0c-045503f010a6/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1431
4862025-01-10T21:54:43.266ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4872025-01-10T21:54:43.266ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4882025-01-10T21:54:43.266ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/598e3879-d23d-4354-9e0c-045503f010a6/crypt/zone/oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42"), full_name: "oxp_598e3879-d23d-4354-9e0c-045503f010a6/crypt/zone/oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42" } file = sled-storage/src/manager.rs:1431
4892025-01-10T21:54:43.325ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4902025-01-10T21:54:43.325ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4912025-01-10T21:54:43.325ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/d27ec511-4e48-4e26-9677-295de0452e91/crypt/zone/oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32"), full_name: "oxp_d27ec511-4e48-4e26-9677-295de0452e91/crypt/zone/oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32" } file = sled-storage/src/manager.rs:1431
4922025-01-10T21:54:43.382ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4932025-01-10T21:54:43.382ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4942025-01-10T21:54:43.382ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_784c4a6b-3804-44fa-afd4-c974bef9dd5d/crucible" } file = sled-storage/src/manager.rs:1431
4952025-01-10T21:54:43.429ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4962025-01-10T21:54:43.429ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4972025-01-10T21:54:43.429ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/73828a50-9586-442b-b526-668d3e1de808/crypt/zone/oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78"), full_name: "oxp_73828a50-9586-442b-b526-668d3e1de808/crypt/zone/oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78" } file = sled-storage/src/manager.rs:1431
4982025-01-10T21:54:43.488ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4992025-01-10T21:54:43.488ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5002025-01-10T21:54:43.488ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/d27ec511-4e48-4e26-9677-295de0452e91/crypt/zone"), full_name: "oxp_d27ec511-4e48-4e26-9677-295de0452e91/crypt/zone" } file = sled-storage/src/manager.rs:1431
5012025-01-10T21:54:43.517ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5022025-01-10T21:54:43.517ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5032025-01-10T21:54:43.517ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/73828a50-9586-442b-b526-668d3e1de808/crypt/zone/oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48"), full_name: "oxp_73828a50-9586-442b-b526-668d3e1de808/crypt/zone/oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48" } file = sled-storage/src/manager.rs:1431
5042025-01-10T21:54:43.577ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5052025-01-10T21:54:43.577ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5062025-01-10T21:54:43.577ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_73828a50-9586-442b-b526-668d3e1de808/crypt/internal_dns" } file = sled-storage/src/manager.rs:1431
5072025-01-10T21:54:43.625ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5082025-01-10T21:54:43.625ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5092025-01-10T21:54:43.625ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/73828a50-9586-442b-b526-668d3e1de808/crypt/zone/oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f"), full_name: "oxp_73828a50-9586-442b-b526-668d3e1de808/crypt/zone/oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f" } file = sled-storage/src/manager.rs:1431
5102025-01-10T21:54:43.684ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5112025-01-10T21:54:43.684ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5122025-01-10T21:54:43.684ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/619e5037-b196-43f6-b092-666b1b1387ff/crypt/zone/oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd"), full_name: "oxp_619e5037-b196-43f6-b092-666b1b1387ff/crypt/zone/oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd" } file = sled-storage/src/manager.rs:1431
5132025-01-10T21:54:43.742ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5142025-01-10T21:54:43.742ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5152025-01-10T21:54:43.742ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/598e3879-d23d-4354-9e0c-045503f010a6/crypt/zone"), full_name: "oxp_598e3879-d23d-4354-9e0c-045503f010a6/crypt/zone" } file = sled-storage/src/manager.rs:1431
5162025-01-10T21:54:43.770ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5172025-01-10T21:54:43.770ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5182025-01-10T21:54:43.770ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_598e3879-d23d-4354-9e0c-045503f010a6/crucible" } file = sled-storage/src/manager.rs:1431
5192025-01-10T21:54:43.787ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/2235d715-657e-4a92-8f59-9a6c0c957b73) file = sled-agent/src/probe_manager.rs:231 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
5202025-01-10T21:54:43.818ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5212025-01-10T21:54:43.818ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5222025-01-10T21:54:43.818ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_5d7ad0a3-e47e-4369-b599-d02dbd3a8829/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1431
5232025-01-10T21:54:43.870ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5242025-01-10T21:54:43.870ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5252025-01-10T21:54:43.870ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/5d7ad0a3-e47e-4369-b599-d02dbd3a8829/crypt/zone/oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487"), full_name: "oxp_5d7ad0a3-e47e-4369-b599-d02dbd3a8829/crypt/zone/oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487" } file = sled-storage/src/manager.rs:1431
5262025-01-10T21:54:43.928ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5272025-01-10T21:54:43.928ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5282025-01-10T21:54:43.928ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/eecab578-d379-46e4-90f5-9dc11839dad5/crypt/zone"), full_name: "oxp_eecab578-d379-46e4-90f5-9dc11839dad5/crypt/zone" } file = sled-storage/src/manager.rs:1431
5292025-01-10T21:54:43.957ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5302025-01-10T21:54:43.957ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5312025-01-10T21:54:43.957ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/7b0e1b76-af4f-49bd-9848-611d8d8666ad/crypt/zone/oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090"), full_name: "oxp_7b0e1b76-af4f-49bd-9848-611d8d8666ad/crypt/zone/oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090" } file = sled-storage/src/manager.rs:1431
5322025-01-10T21:54:44.015ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5332025-01-10T21:54:44.015ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5342025-01-10T21:54:44.015ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_ce5c629d-252d-4e12-9f9d-2087cb772d81/crucible" } file = sled-storage/src/manager.rs:1431
5352025-01-10T21:54:44.062ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5362025-01-10T21:54:44.062ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5372025-01-10T21:54:44.062ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/73828a50-9586-442b-b526-668d3e1de808/crypt/zone"), full_name: "oxp_73828a50-9586-442b-b526-668d3e1de808/crypt/zone" } file = sled-storage/src/manager.rs:1431
5382025-01-10T21:54:44.090ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5392025-01-10T21:54:44.090ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5402025-01-10T21:54:44.090ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_ce5c629d-252d-4e12-9f9d-2087cb772d81/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1431
5412025-01-10T21:54:44.139ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5422025-01-10T21:54:44.139ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5432025-01-10T21:54:44.139ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_73828a50-9586-442b-b526-668d3e1de808/crypt/clickhouse" } file = sled-storage/src/manager.rs:1431
5442025-01-10T21:54:44.189ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5452025-01-10T21:54:44.189ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5462025-01-10T21:54:44.189ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/784c4a6b-3804-44fa-afd4-c974bef9dd5d/crypt/zone"), full_name: "oxp_784c4a6b-3804-44fa-afd4-c974bef9dd5d/crypt/zone" } file = sled-storage/src/manager.rs:1431
5472025-01-10T21:54:44.218ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5482025-01-10T21:54:44.218ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5492025-01-10T21:54:44.218ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/784c4a6b-3804-44fa-afd4-c974bef9dd5d/crypt/zone/oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047"), full_name: "oxp_784c4a6b-3804-44fa-afd4-c974bef9dd5d/crypt/zone/oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047" } file = sled-storage/src/manager.rs:1431
5502025-01-10T21:54:44.276ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5512025-01-10T21:54:44.277ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5522025-01-10T21:54:44.277ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_73828a50-9586-442b-b526-668d3e1de808/crucible" } file = sled-storage/src/manager.rs:1431
5532025-01-10T21:54:44.323ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5542025-01-10T21:54:44.323ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5552025-01-10T21:54:44.323ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/eecab578-d379-46e4-90f5-9dc11839dad5/crypt/zone/oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6"), full_name: "oxp_eecab578-d379-46e4-90f5-9dc11839dad5/crypt/zone/oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6" } file = sled-storage/src/manager.rs:1431
5562025-01-10T21:54:44.382ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5572025-01-10T21:54:44.382ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5582025-01-10T21:54:44.382ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/73828a50-9586-442b-b526-668d3e1de808/crypt/zone/oxz_external_dns_bce93894-738c-4596-b983-281da931e977"), full_name: "oxp_73828a50-9586-442b-b526-668d3e1de808/crypt/zone/oxz_external_dns_bce93894-738c-4596-b983-281da931e977" } file = sled-storage/src/manager.rs:1431
5592025-01-10T21:54:44.443ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5602025-01-10T21:54:44.443ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5612025-01-10T21:54:44.443ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/ce5c629d-252d-4e12-9f9d-2087cb772d81/crypt/zone/oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612"), full_name: "oxp_ce5c629d-252d-4e12-9f9d-2087cb772d81/crypt/zone/oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612" } file = sled-storage/src/manager.rs:1431
5622025-01-10T21:54:44.504ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5632025-01-10T21:54:44.504ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5642025-01-10T21:54:44.504ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/ce5c629d-252d-4e12-9f9d-2087cb772d81/crypt/zone"), full_name: "oxp_ce5c629d-252d-4e12-9f9d-2087cb772d81/crypt/zone" } file = sled-storage/src/manager.rs:1431
5652025-01-10T21:54:44.532ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5662025-01-10T21:54:44.532ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5672025-01-10T21:54:44.532ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/619e5037-b196-43f6-b092-666b1b1387ff/crypt/zone"), full_name: "oxp_619e5037-b196-43f6-b092-666b1b1387ff/crypt/zone" } file = sled-storage/src/manager.rs:1431
5682025-01-10T21:54:44.561ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5692025-01-10T21:54:44.561ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5702025-01-10T21:54:44.561ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/598e3879-d23d-4354-9e0c-045503f010a6/crypt/debug"), full_name: "oxp_598e3879-d23d-4354-9e0c-045503f010a6/crypt/debug" } file = sled-storage/src/manager.rs:1431
5712025-01-10T21:54:44.589ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5722025-01-10T21:54:44.589ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5732025-01-10T21:54:44.589ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/7b0e1b76-af4f-49bd-9848-611d8d8666ad/crypt/debug"), full_name: "oxp_7b0e1b76-af4f-49bd-9848-611d8d8666ad/crypt/debug" } file = sled-storage/src/manager.rs:1431
5742025-01-10T21:54:44.616ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5752025-01-10T21:54:44.616ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5762025-01-10T21:54:44.616ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/5d7ad0a3-e47e-4369-b599-d02dbd3a8829/crypt/zone/oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b"), full_name: "oxp_5d7ad0a3-e47e-4369-b599-d02dbd3a8829/crypt/zone/oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b" } file = sled-storage/src/manager.rs:1431
5772025-01-10T21:54:44.675ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5782025-01-10T21:54:44.675ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5792025-01-10T21:54:44.675ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/eecab578-d379-46e4-90f5-9dc11839dad5/crypt/zone/oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f"), full_name: "oxp_eecab578-d379-46e4-90f5-9dc11839dad5/crypt/zone/oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f" } file = sled-storage/src/manager.rs:1431
5802025-01-10T21:54:44.734ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5812025-01-10T21:54:44.734ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5822025-01-10T21:54:44.734ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/619e5037-b196-43f6-b092-666b1b1387ff/crypt/debug"), full_name: "oxp_619e5037-b196-43f6-b092-666b1b1387ff/crypt/debug" } file = sled-storage/src/manager.rs:1431
5832025-01-10T21:54:44.762ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5842025-01-10T21:54:44.762ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5852025-01-10T21:54:44.762ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_5d7ad0a3-e47e-4369-b599-d02dbd3a8829/crucible" } file = sled-storage/src/manager.rs:1431
5862025-01-10T21:54:44.788ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/2235d715-657e-4a92-8f59-9a6c0c957b73) file = sled-agent/src/probe_manager.rs:231 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
5872025-01-10T21:54:44.814ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5882025-01-10T21:54:44.814ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5892025-01-10T21:54:44.814ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/ce5c629d-252d-4e12-9f9d-2087cb772d81/crypt/zone/oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d"), full_name: "oxp_ce5c629d-252d-4e12-9f9d-2087cb772d81/crypt/zone/oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d" } file = sled-storage/src/manager.rs:1431
5902025-01-10T21:54:44.874ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5912025-01-10T21:54:44.874ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5922025-01-10T21:54:44.874ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/5d7ad0a3-e47e-4369-b599-d02dbd3a8829/crypt/zone/oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066"), full_name: "oxp_5d7ad0a3-e47e-4369-b599-d02dbd3a8829/crypt/zone/oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066" } file = sled-storage/src/manager.rs:1431
5932025-01-10T21:54:44.934ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5942025-01-10T21:54:44.934ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5952025-01-10T21:54:44.934ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/73828a50-9586-442b-b526-668d3e1de808/crypt/zone/oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2"), full_name: "oxp_73828a50-9586-442b-b526-668d3e1de808/crypt/zone/oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2" } file = sled-storage/src/manager.rs:1431
5962025-01-10T21:54:44.993ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5972025-01-10T21:54:44.993ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5982025-01-10T21:54:44.993ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_73828a50-9586-442b-b526-668d3e1de808/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1431
5992025-01-10T21:54:45.043ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6002025-01-10T21:54:45.043ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6012025-01-10T21:54:45.043ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_d27ec511-4e48-4e26-9677-295de0452e91/crucible" } file = sled-storage/src/manager.rs:1431
6022025-01-10T21:54:45.093ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6032025-01-10T21:54:45.093ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
6042025-01-10T21:54:45.093ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/d27ec511-4e48-4e26-9677-295de0452e91/crypt/debug"), full_name: "oxp_d27ec511-4e48-4e26-9677-295de0452e91/crypt/debug" } file = sled-storage/src/manager.rs:1431
6052025-01-10T21:54:45.121ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6062025-01-10T21:54:45.121ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
6072025-01-10T21:54:45.121ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/7b0e1b76-af4f-49bd-9848-611d8d8666ad/crypt/zone"), full_name: "oxp_7b0e1b76-af4f-49bd-9848-611d8d8666ad/crypt/zone" } file = sled-storage/src/manager.rs:1431
6082025-01-10T21:54:45.150ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6092025-01-10T21:54:45.150ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6102025-01-10T21:54:45.150ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/598e3879-d23d-4354-9e0c-045503f010a6/crypt/zone/oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc"), full_name: "oxp_598e3879-d23d-4354-9e0c-045503f010a6/crypt/zone/oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc" } file = sled-storage/src/manager.rs:1431
6112025-01-10T21:54:45.211ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6122025-01-10T21:54:45.211ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6132025-01-10T21:54:45.211ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_d27ec511-4e48-4e26-9677-295de0452e91/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1431
6142025-01-10T21:54:45.261ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6152025-01-10T21:54:45.261ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6162025-01-10T21:54:45.261ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/784c4a6b-3804-44fa-afd4-c974bef9dd5d/crypt/zone/oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935"), full_name: "oxp_784c4a6b-3804-44fa-afd4-c974bef9dd5d/crypt/zone/oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935" } file = sled-storage/src/manager.rs:1431
6172025-01-10T21:54:45.323ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6182025-01-10T21:54:45.323ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6192025-01-10T21:54:45.323ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/ce5c629d-252d-4e12-9f9d-2087cb772d81/crypt/zone/oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50"), full_name: "oxp_ce5c629d-252d-4e12-9f9d-2087cb772d81/crypt/zone/oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50" } file = sled-storage/src/manager.rs:1431
6202025-01-10T21:54:45.382ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6212025-01-10T21:54:45.382ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6222025-01-10T21:54:45.382ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/598e3879-d23d-4354-9e0c-045503f010a6/crypt/zone/oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052"), full_name: "oxp_598e3879-d23d-4354-9e0c-045503f010a6/crypt/zone/oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052" } file = sled-storage/src/manager.rs:1431
6232025-01-10T21:54:45.442ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6242025-01-10T21:54:45.442ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6252025-01-10T21:54:45.442ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/598e3879-d23d-4354-9e0c-045503f010a6/crypt/zone/oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30"), full_name: "oxp_598e3879-d23d-4354-9e0c-045503f010a6/crypt/zone/oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30" } file = sled-storage/src/manager.rs:1431
6262025-01-10T21:54:45.500ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6272025-01-10T21:54:45.500ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6282025-01-10T21:54:45.500ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/598e3879-d23d-4354-9e0c-045503f010a6/crypt/zone/oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a"), full_name: "oxp_598e3879-d23d-4354-9e0c-045503f010a6/crypt/zone/oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a" } file = sled-storage/src/manager.rs:1431
6292025-01-10T21:54:45.559ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6302025-01-10T21:54:45.559ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6312025-01-10T21:54:45.559ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/619e5037-b196-43f6-b092-666b1b1387ff/crypt/zone/oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f"), full_name: "oxp_619e5037-b196-43f6-b092-666b1b1387ff/crypt/zone/oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f" } file = sled-storage/src/manager.rs:1431
6322025-01-10T21:54:45.620ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6332025-01-10T21:54:45.620ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6342025-01-10T21:54:45.620ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/598e3879-d23d-4354-9e0c-045503f010a6/crypt/zone/oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594"), full_name: "oxp_598e3879-d23d-4354-9e0c-045503f010a6/crypt/zone/oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594" } file = sled-storage/src/manager.rs:1431
6352025-01-10T21:54:45.682ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6362025-01-10T21:54:45.682ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6372025-01-10T21:54:45.682ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_619e5037-b196-43f6-b092-666b1b1387ff/crucible" } file = sled-storage/src/manager.rs:1431
6382025-01-10T21:54:45.731ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6392025-01-10T21:54:45.731ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
6402025-01-10T21:54:45.731ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/eecab578-d379-46e4-90f5-9dc11839dad5/crypt/debug"), full_name: "oxp_eecab578-d379-46e4-90f5-9dc11839dad5/crypt/debug" } file = sled-storage/src/manager.rs:1431
6412025-01-10T21:54:45.760ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6422025-01-10T21:54:45.760ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6432025-01-10T21:54:45.760ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_5d7ad0a3-e47e-4369-b599-d02dbd3a8829/crypt/internal_dns" } file = sled-storage/src/manager.rs:1431
6442025-01-10T21:54:45.793ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/2235d715-657e-4a92-8f59-9a6c0c957b73) file = sled-agent/src/probe_manager.rs:231 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
6452025-01-10T21:54:45.812ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6462025-01-10T21:54:45.812ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6472025-01-10T21:54:45.812ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_598e3879-d23d-4354-9e0c-045503f010a6/crypt/internal_dns" } file = sled-storage/src/manager.rs:1431
6482025-01-10T21:54:45.862ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6492025-01-10T21:54:45.862ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6502025-01-10T21:54:45.862ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_eecab578-d379-46e4-90f5-9dc11839dad5/crucible" } file = sled-storage/src/manager.rs:1431
6512025-01-10T21:54:45.910ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6522025-01-10T21:54:45.910ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6532025-01-10T21:54:45.910ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_73828a50-9586-442b-b526-668d3e1de808/crypt/external_dns" } file = sled-storage/src/manager.rs:1431
6542025-01-10T21:54:45.960ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/.omicron-datasets.json.tmp file = common/src/ledger.rs:199 request = datasets_ensure
6552025-01-10T21:54:45.960ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/10481327-13d1-4548-950f-130d063e2690/config/.omicron-datasets.json.tmp file = common/src/ledger.rs:199 request = datasets_ensure
6562025-01-10T21:54:45.960ZINFOSledAgent: datasets ensure: Updated storage file = sled-agent/src/sled_agent.rs:861 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
6572025-01-10T21:54:45.960ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3180593 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:35668 req_id = 353441d1-9cce-45a2-abfb-cc5c22de8f66 response_code = 200 uri = /datasets
6582025-01-10T21:54:45.992ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: e59b9a62-bb9f-4749-8a33-85254995194f (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 282e2202-986b-4d4d-a7ee-e57d0fc63594 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 6f2bb3b2-aa25-411c-817d-fd4c720ec487 (service), filesystem_pool: Some(ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
6592025-01-10T21:54:45.992ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62421
6602025-01-10T21:54:45.993ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
6612025-01-10T21:54:45.993ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6622025-01-10T21:54:46.001ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
6632025-01-10T21:54:46.001ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6642025-01-10T21:54:46.010ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
6652025-01-10T21:54:46.010ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6662025-01-10T21:54:46.019ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
6672025-01-10T21:54:46.019ZINFOSledAgent (ServiceManager): No ledger in /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-zones.json file = common/src/ledger.rs:188
6682025-01-10T21:54:46.019ZINFOSledAgent (ServiceManager): No ledger in /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-zones.json file = common/src/ledger.rs:188
6692025-01-10T21:54:46.019ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
6702025-01-10T21:54:46.063ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
6712025-01-10T21:54:46.063ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
6722025-01-10T21:54:46.063ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
6732025-01-10T21:54:46.114ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f file = illumos-utils/src/zone.rs:325
6742025-01-10T21:54:46.117ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487 file = illumos-utils/src/zone.rs:325
6752025-01-10T21:54:46.121ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594 file = illumos-utils/src/zone.rs:325
6762025-01-10T21:54:46.132ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f file = illumos-utils/src/zone.rs:363
6772025-01-10T21:54:46.137ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487 file = illumos-utils/src/zone.rs:363
6782025-01-10T21:54:46.141ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594 file = illumos-utils/src/zone.rs:363
6792025-01-10T21:54:46.795ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/2235d715-657e-4a92-8f59-9a6c0c957b73) file = sled-agent/src/probe_manager.rs:231 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
6802025-01-10T21:54:47.766ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService2'/> <propval type="astring" name="gateway" value='fd00:1122:3344:3::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:3::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:3::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:3::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6812025-01-10T21:54:47.766ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:3::, width: 64 }
6822025-01-10T21:54:47.766ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487
6832025-01-10T21:54:48.202ZINFOSledAgent (ServiceManager): refreshed lldpd service with new configuration file = sled-agent/src/services.rs:4507
6842025-01-10T21:54:48.202ZINFOSledAgent (ServiceManager): configuring MGS service file = sled-agent/src/services.rs:4353
6852025-01-10T21:54:48.257ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService3'/> <propval type="astring" name="gateway" value='fd00:1122:3344:2::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:2::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:2::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:2::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6862025-01-10T21:54:48.257ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:2::, width: 64 }
6872025-01-10T21:54:48.270ZINFOSledAgent (ServiceManager): refreshed MGS service with new configuration file = sled-agent/src/services.rs:4395
6882025-01-10T21:54:48.270ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:4474 rack_subnet = fd00:1122:3344::
6892025-01-10T21:54:48.294ZINFOSledAgent (ServiceManager): refreshed wicketd service with new configuration file = sled-agent/src/services.rs:4485
6902025-01-10T21:54:48.294ZINFOSledAgent (ServiceManager): configuring mgd service file = sled-agent/src/services.rs:4527
6912025-01-10T21:54:48.365ZINFOSledAgent (ServiceManager): refreshed mgd service with new configuration file = sled-agent/src/services.rs:4559
6922025-01-10T21:54:48.365ZINFOSledAgent (ServiceManager): configuring mg-ddm service file = sled-agent/src/services.rs:4565
6932025-01-10T21:54:48.455ZINFOSledAgent (ServiceManager): refreshed mg-ddm service with new configuration file = sled-agent/src/services.rs:4606
6942025-01-10T21:54:48.455ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:370
6952025-01-10T21:54:48.534ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 116895860409675312 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
6962025-01-10T21:54:48.534ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 4457687850656690512 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
6972025-01-10T21:54:48.683ZWARNSledAgent (ServiceManager): Failed to get switch ID from MGS (retrying in 63.800446ms) error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::2]:12225/local/switch-id) file = sled-agent/src/bootstrap/early_networking.rs:389
6982025-01-10T21:54:48.747ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService1'/> <propval type="astring" name="gateway" value='fd00:1122:3344:1::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:1::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:1::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:1::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6992025-01-10T21:54:48.751ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:420
7002025-01-10T21:54:48.755ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487") failed: Property not found. retry in 70.864384ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487
7012025-01-10T21:54:48.755ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594
7022025-01-10T21:54:48.760ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:1::, width: 64 }
7032025-01-10T21:54:48.760ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f
7042025-01-10T21:54:48.797ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:717
7052025-01-10T21:54:48.797ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:762
7062025-01-10T21:54:48.797ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:765 uptime = 0
7072025-01-10T21:54:48.797ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.2.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: Some(\n None,\n ),\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n tx_eq: None,\n },\n },\n },\n} file = sled-agent/src/bootstrap/early_networking.rs:444
7082025-01-10T21:54:48.842ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
7092025-01-10T21:54:48.844ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487") failed: Property not found. retry in 58.042228ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487
7102025-01-10T21:54:48.899ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594") failed: Property not found. retry in 34.844556ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594
7112025-01-10T21:54:48.899ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:607
7122025-01-10T21:54:48.901ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
7132025-01-10T21:54:48.901ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
7142025-01-10T21:54:48.901ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
7152025-01-10T21:54:48.901ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
7162025-01-10T21:54:48.901ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
7172025-01-10T21:54:48.901ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
7182025-01-10T21:54:48.901ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
7192025-01-10T21:54:48.901ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
7202025-01-10T21:54:48.901ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
7212025-01-10T21:54:48.901ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
7222025-01-10T21:54:48.901ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
7232025-01-10T21:54:48.901ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
7242025-01-10T21:54:48.901ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
7252025-01-10T21:54:48.902ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
7262025-01-10T21:54:48.902ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
7272025-01-10T21:54:48.902ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
7282025-01-10T21:54:48.902ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
7292025-01-10T21:54:48.902ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
7302025-01-10T21:54:48.902ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
7312025-01-10T21:54:48.902ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
7322025-01-10T21:54:48.902ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
7332025-01-10T21:54:48.902ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
7342025-01-10T21:54:48.906ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f") failed: Property not found. retry in 45.13898ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f
7352025-01-10T21:54:48.916ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487") failed: Property not found. retry in 244.883245ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487
7362025-01-10T21:54:48.942ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594") failed: Property not found. retry in 108.937756ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594
7372025-01-10T21:54:48.960ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f") failed: Property not found. retry in 122.10407ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f
7382025-01-10T21:54:49.110ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594") failed: Property not found. retry in 167.229257ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594
7392025-01-10T21:54:49.218ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f") failed: Property not found. retry in 277.166712ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f
7402025-01-10T21:54:49.256ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487") failed: Property not found. retry in 594.511397ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487
7412025-01-10T21:54:49.316ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594") failed: Property not found. retry in 297.121951ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594
7422025-01-10T21:54:49.509ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f") failed: Property not found. retry in 303.732584ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f
7432025-01-10T21:54:49.625ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594") failed: Property not found. retry in 845.762685ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594
7442025-01-10T21:54:49.778ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 13884378162807884192 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
7452025-01-10T21:54:49.778ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17833868550910882914 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
7462025-01-10T21:54:49.821ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f") failed: Property not found. retry in 794.543581ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f
7472025-01-10T21:54:49.859ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487") failed: Property not found. retry in 978.261333ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487
7482025-01-10T21:54:50.481ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594") failed: Property not found. retry in 1.129071292s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594
7492025-01-10T21:54:50.626ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f") failed: Property not found. retry in 1.391023878s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f
7502025-01-10T21:54:50.845ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487") failed: Property not found. retry in 1.133566428s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487
7512025-01-10T21:54:51.619ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594") failed: Property not found. retry in 967.301553ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594
7522025-01-10T21:54:51.988ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487") failed: Property not found. retry in 1.010810044s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487
7532025-01-10T21:54:52.026ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f") failed: Property not found. retry in 947.043242ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f
7542025-01-10T21:54:52.596ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594") failed: Property not found. retry in 861.675613ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594
7552025-01-10T21:54:52.982ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f") failed: Property not found. retry in 588.395908ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f
7562025-01-10T21:54:53.007ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487") failed: Property not found. retry in 1.084021861s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487
7572025-01-10T21:54:53.466ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594") failed: Property not found. retry in 1.256631802s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594
7582025-01-10T21:54:53.579ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f") failed: Property not found. retry in 1.317621279s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f
7592025-01-10T21:54:54.099ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487") failed: Property not found. retry in 1.190021096s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487
7602025-01-10T21:54:54.732ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594") failed: Property not found. retry in 1.370223454s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594
7612025-01-10T21:54:54.906ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f") failed: Property not found. retry in 803.508535ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f
7622025-01-10T21:54:55.719ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f") failed: Property not found. retry in 790.708871ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f
7632025-01-10T21:54:56.530ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_internal_dns_e59b9a62-bb9f-4749-8a33-85254995194f
7642025-01-10T21:54:56.530ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_internal_dns_6f2bb3b2-aa25-411c-817d-fd4c720ec487
7652025-01-10T21:54:56.530ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_internal_dns_282e2202-986b-4d4d-a7ee-e57d0fc63594
7662025-01-10T21:54:56.530ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7672025-01-10T21:54:56.530ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/10481327-13d1-4548-950f-130d063e2690/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7682025-01-10T21:54:56.530ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 10537986 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:62421 req_id = f7431485-407d-4953-a1f7-dae4c7005a47 response_code = 204 uri = /omicron-zones
7692025-01-10T21:54:56.533ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:658
7702025-01-10T21:54:58.440ZWARNSledAgent (RSS): failed to write DNS configuration (will retry in 116.53197ms) dns_config_addr = [fd00:1122:3344:1::1]:5353 error_message = Communication Error: error sending request for url (http://[fd00:1122:3344:1::1]:5353/config) file = sled-agent/src/rack_setup/service.rs:678
7712025-01-10T21:54:58.536ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 116895860409675312 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
7722025-01-10T21:54:58.536ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 4457687850656690512 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
7732025-01-10T21:54:58.668ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:658
7742025-01-10T21:54:58.779ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:658
7752025-01-10T21:54:58.845ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:694
7762025-01-10T21:54:58.845ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
7772025-01-10T21:54:58.875ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
7782025-01-10T21:54:58.875ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
7792025-01-10T21:54:58.875ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
7802025-01-10T21:54:58.904ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: e59b9a62-bb9f-4749-8a33-85254995194f (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 282e2202-986b-4d4d-a7ee-e57d0fc63594 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 6f2bb3b2-aa25-411c-817d-fd4c720ec487 (service), filesystem_pool: Some(ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 04fec8c4-c1f3-4609-8a2e-2fbb5b456603 (service), filesystem_pool: Some(ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 96f2491a-6c10-433d-a3d4-7d45935ab973, kind: Service { id: 04fec8c4-c1f3-4609-8a2e-2fbb5b456603 }, name: Name("ntp-04fec8c4-c1f3-4609-8a2e-2fbb5b456603"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 48])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
7812025-01-10T21:54:58.904ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57379
7822025-01-10T21:54:58.905ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
7832025-01-10T21:54:58.905ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7842025-01-10T21:54:58.915ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
7852025-01-10T21:54:58.915ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7862025-01-10T21:54:58.925ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
7872025-01-10T21:54:58.925ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7882025-01-10T21:54:58.935ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
7892025-01-10T21:54:58.935ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-zones.json file = common/src/ledger.rs:177
7902025-01-10T21:54:58.936ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-zones.json file = common/src/ledger.rs:177
7912025-01-10T21:54:58.936ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
7922025-01-10T21:54:58.948ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
7932025-01-10T21:54:58.981ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
7942025-01-10T21:54:58.982ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
7952025-01-10T21:54:58.982ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
7962025-01-10T21:54:59.012ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte0", ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 179, 48]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 } }) }
7972025-01-10T21:54:59.012ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1275 zone_type = boundary_ntp
7982025-01-10T21:54:59.012ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:83
7992025-01-10T21:54:59.012ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:90
8002025-01-10T21:54:59.012ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:118
8012025-01-10T21:54:59.013ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
8022025-01-10T21:54:59.037ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603 file = illumos-utils/src/zone.rs:325
8032025-01-10T21:54:59.056ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603 file = illumos-utils/src/zone.rs:363
8042025-01-10T21:54:59.682ZINFOSledAgent (ServiceManager): Profile for oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService4'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/chrony-setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="allow" value='fd00:1122:3344:100::/56'/> <propval type="boolean" name="boundary" value='true'/> <propval type="astring" name="boundary_pool" value='boundary-ntp.control-plane.oxide.internal'/> <propval type="astring" name="server" value='0.pool.ntp.org'/> </property_group> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="1.1.1.1" /> <value_node value="9.9.9.9" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/ntp"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte0'/> <propval type="astring" name="gateway" value='172.30.3.1'/> <propval type="astring" name="ip" value='172.30.3.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8052025-01-10T21:54:59.682ZINFOSledAgent (ServiceManager): Booting oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603
8062025-01-10T21:54:59.780ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 13884378162807884192 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
8072025-01-10T21:54:59.780ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17833868550910882914 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
8082025-01-10T21:54:59.818ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603") failed: Property not found. retry in 25.695549ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603
8092025-01-10T21:54:59.851ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603") failed: Property not found. retry in 107.997342ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603
8102025-01-10T21:54:59.971ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603") failed: Property not found. retry in 275.135824ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603
8112025-01-10T21:55:00.258ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603") failed: Property not found. retry in 270.792605ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603
8122025-01-10T21:55:00.538ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603") failed: Property not found. retry in 895.237662ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603
8132025-01-10T21:55:01.443ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603") failed: Property not found. retry in 613.258837ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603
8142025-01-10T21:55:02.066ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603") failed: Property not found. retry in 1.247790736s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603
8152025-01-10T21:55:02.801ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/2235d715-657e-4a92-8f59-9a6c0c957b73) file = sled-agent/src/probe_manager.rs:231 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
8162025-01-10T21:55:03.113ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 13.850317526s error = "no record found for Query { name: Name(\\"_nexus._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }" file = oximeter/producer/src/lib.rs:387 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
8172025-01-10T21:55:03.322ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603") failed: Property not found. retry in 1.210133743s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603
8182025-01-10T21:55:03.493ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/2235d715-657e-4a92-8f59-9a6c0c957b73) file = sled-agent/src/nexus.rs:527 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
8192025-01-10T21:55:04.542ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603") failed: Property not found. retry in 569.46155ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603
8202025-01-10T21:55:05.129ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603
8212025-01-10T21:55:05.129ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
8222025-01-10T21:55:05.130ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/10481327-13d1-4548-950f-130d063e2690/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
8232025-01-10T21:55:05.130ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6225359 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:57379 req_id = 40780ac6-e82b-4bfd-aeda-85fc13f59c9a response_code = 204 uri = /omicron-zones
8242025-01-10T21:55:05.133ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:735
8252025-01-10T21:55:05.162ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8262025-01-10T21:55:05.162ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:52093
8272025-01-10T21:55:05.172ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3926
8282025-01-10T21:55:05.172ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 9593 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:52093 req_id = d57737de-cb7d-43c7-9502-94120fd883fc response_code = 500 uri = /timesync
8292025-01-10T21:55:05.175ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8302025-01-10T21:55:05.282ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8312025-01-10T21:55:05.282ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50696
8322025-01-10T21:55:05.313ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 15585254935996880898 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
8332025-01-10T21:55:06.125ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 2497563704124902091 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
8342025-01-10T21:55:06.533ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 9335681955878478700 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
8352025-01-10T21:55:08.538ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 116895860409675312 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
8362025-01-10T21:55:08.538ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 4457687850656690512 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
8372025-01-10T21:55:09.291ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3926
8382025-01-10T21:55:09.291ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 4008993 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:50696 req_id = 4f1b400b-7db6-4f23-b42e-ff9fd6e21321 response_code = 500 uri = /timesync
8392025-01-10T21:55:09.294ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8402025-01-10T21:55:09.387ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8412025-01-10T21:55:09.387ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:37121
8422025-01-10T21:55:09.396ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3926
8432025-01-10T21:55:09.396ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 8696 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:37121 req_id = a3cd1aa7-2ac5-426d-8567-26078db18e27 response_code = 500 uri = /timesync
8442025-01-10T21:55:09.399ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8452025-01-10T21:55:09.629ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8462025-01-10T21:55:09.629ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:56618
8472025-01-10T21:55:09.636ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3926
8482025-01-10T21:55:09.637ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 7645 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:56618 req_id = fa1645b9-b7ee-4036-8ad0-d281351a0748 response_code = 500 uri = /timesync
8492025-01-10T21:55:09.640ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8502025-01-10T21:55:09.782ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 13884378162807884192 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
8512025-01-10T21:55:09.782ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17833868550910882914 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
8522025-01-10T21:55:09.925ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8532025-01-10T21:55:09.925ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:52545
8542025-01-10T21:55:10.933ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_04fec8c4-c1f3-4609-8a2e-2fbb5b456603': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3926
8552025-01-10T21:55:10.933ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 1008779 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:52545 req_id = 4a9477a3-c4b4-4c8f-b371-3fa048ee05e7 response_code = 500 uri = /timesync
8562025-01-10T21:55:10.936ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8572025-01-10T21:55:11.188ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8582025-01-10T21:55:11.188ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:34015
8592025-01-10T21:55:11.196ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7691 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:34015 req_id = 5a252fb3-9d92-4f4b-965f-de430e183684 response_code = 200 uri = /timesync
8602025-01-10T21:55:11.198ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:743
8612025-01-10T21:55:11.198ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8622025-01-10T21:55:11.449ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8632025-01-10T21:55:11.449ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60732
8642025-01-10T21:55:11.457ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7739 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:60732 req_id = 5f5bb7b2-9e41-4988-8adf-6d937622f1a7 response_code = 200 uri = /timesync
8652025-01-10T21:55:11.460ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:743
8662025-01-10T21:55:11.460ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8672025-01-10T21:55:11.721ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8682025-01-10T21:55:11.721ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:52745
8692025-01-10T21:55:11.727ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6522 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:52745 req_id = 78df8143-0e1f-4664-9a03-c77dfe273211 response_code = 200 uri = /timesync
8702025-01-10T21:55:11.730ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:743
8712025-01-10T21:55:11.730ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8722025-01-10T21:55:12.206ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8732025-01-10T21:55:12.206ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:53751
8742025-01-10T21:55:12.214ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7960 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:53751 req_id = d30474ed-621b-4516-b2e2-cd25225b3b68 response_code = 200 uri = /timesync
8752025-01-10T21:55:12.215ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:743
8762025-01-10T21:55:12.215ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8772025-01-10T21:55:12.751ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8782025-01-10T21:55:12.751ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:42698
8792025-01-10T21:55:13.016ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 18094 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:42698 req_id = 75789b0d-5153-4681-940c-82a294d0b69b response_code = 200 uri = /timesync
8802025-01-10T21:55:13.019ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 2429174486, ip_addr: 144.202.66.214, stratum: 4, ref_time: 1736546112.7148335, correction: 0.0 } file = sled-agent/src/rack_setup/service.rs:743
8812025-01-10T21:55:13.019ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1392
8822025-01-10T21:55:13.049ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: e59b9a62-bb9f-4749-8a33-85254995194f (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 282e2202-986b-4d4d-a7ee-e57d0fc63594 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 6f2bb3b2-aa25-411c-817d-fd4c720ec487 (service), filesystem_pool: Some(ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 04fec8c4-c1f3-4609-8a2e-2fbb5b456603 (service), filesystem_pool: Some(ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 96f2491a-6c10-433d-a3d4-7d45935ab973, kind: Service { id: 04fec8c4-c1f3-4609-8a2e-2fbb5b456603 }, name: Name("ntp-04fec8c4-c1f3-4609-8a2e-2fbb5b456603"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 48])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 1b973bfa-6720-4fea-927a-5007c3c3fd48 (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } } } }, OmicronZoneConfig { id: 57b13ff2-7f23-49d7-93f9-9e02946b4c42 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9d57690c-8d29-49b6-a53d-5133c512e066 (service), filesystem_pool: Some(ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4a1ebfac-ae5e-4868-980a-299896e9fc32 (service), filesystem_pool: Some(ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External } } } }, OmicronZoneConfig { id: 80d005ab-2ce7-46d4-adc4-041a00673c50 (service), filesystem_pool: Some(ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
8832025-01-10T21:55:13.049ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:42468
8842025-01-10T21:55:13.049ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
8852025-01-10T21:55:13.049ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8862025-01-10T21:55:13.058ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
8872025-01-10T21:55:13.058ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8882025-01-10T21:55:13.068ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
8892025-01-10T21:55:13.068ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8902025-01-10T21:55:13.076ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
8912025-01-10T21:55:13.076ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8922025-01-10T21:55:13.084ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
8932025-01-10T21:55:13.084ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8942025-01-10T21:55:13.093ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
8952025-01-10T21:55:13.093ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8962025-01-10T21:55:13.103ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
8972025-01-10T21:55:13.103ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8982025-01-10T21:55:13.112ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
8992025-01-10T21:55:13.112ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9002025-01-10T21:55:13.120ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9012025-01-10T21:55:13.121ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-zones.json file = common/src/ledger.rs:177
9022025-01-10T21:55:13.121ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-zones.json file = common/src/ledger.rs:177
9032025-01-10T21:55:13.121ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9042025-01-10T21:55:13.152ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
9052025-01-10T21:55:13.152ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
9062025-01-10T21:55:13.152ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
9072025-01-10T21:55:13.152ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
9082025-01-10T21:55:13.152ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
9092025-01-10T21:55:13.152ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50", TimeNotSynchronized), ("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066", TimeNotSynchronized), ("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32", TimeNotSynchronized), ("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42", TimeNotSynchronized), ("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 103088 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:42468 req_id = 90672f35-7bd7-44ff-a2a6-751f36ec1b02 response_code = 500 uri = /omicron-zones
9102025-01-10T21:55:13.152ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "90672f35-7bd7-44ff-a2a6-751f36ec1b02", "content-length": "124", "date": "Fri, 10 Jan 2025 21:55:12 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "90672f35-7bd7-44ff-a2a6-751f36ec1b02" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 58.108682ms sled_address = [fd00:1122:3344:101::1]:12345
9112025-01-10T21:55:13.212ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: e59b9a62-bb9f-4749-8a33-85254995194f (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 282e2202-986b-4d4d-a7ee-e57d0fc63594 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 6f2bb3b2-aa25-411c-817d-fd4c720ec487 (service), filesystem_pool: Some(ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 04fec8c4-c1f3-4609-8a2e-2fbb5b456603 (service), filesystem_pool: Some(ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 96f2491a-6c10-433d-a3d4-7d45935ab973, kind: Service { id: 04fec8c4-c1f3-4609-8a2e-2fbb5b456603 }, name: Name("ntp-04fec8c4-c1f3-4609-8a2e-2fbb5b456603"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 48])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 1b973bfa-6720-4fea-927a-5007c3c3fd48 (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } } } }, OmicronZoneConfig { id: 57b13ff2-7f23-49d7-93f9-9e02946b4c42 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9d57690c-8d29-49b6-a53d-5133c512e066 (service), filesystem_pool: Some(ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4a1ebfac-ae5e-4868-980a-299896e9fc32 (service), filesystem_pool: Some(ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External } } } }, OmicronZoneConfig { id: 80d005ab-2ce7-46d4-adc4-041a00673c50 (service), filesystem_pool: Some(ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
9122025-01-10T21:55:13.212ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9132025-01-10T21:55:13.212ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9142025-01-10T21:55:13.221ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9152025-01-10T21:55:13.221ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9162025-01-10T21:55:13.229ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9172025-01-10T21:55:13.229ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9182025-01-10T21:55:13.239ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9192025-01-10T21:55:13.239ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9202025-01-10T21:55:13.248ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9212025-01-10T21:55:13.248ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9222025-01-10T21:55:13.257ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9232025-01-10T21:55:13.257ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9242025-01-10T21:55:13.265ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9252025-01-10T21:55:13.265ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9262025-01-10T21:55:13.275ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9272025-01-10T21:55:13.275ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9282025-01-10T21:55:13.284ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9292025-01-10T21:55:13.284ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-zones.json file = common/src/ledger.rs:177
9302025-01-10T21:55:13.284ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-zones.json file = common/src/ledger.rs:177
9312025-01-10T21:55:13.284ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9322025-01-10T21:55:13.313ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
9332025-01-10T21:55:13.314ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
9342025-01-10T21:55:13.314ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
9352025-01-10T21:55:13.314ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
9362025-01-10T21:55:13.314ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
9372025-01-10T21:55:13.314ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48", TimeNotSynchronized), ("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42", TimeNotSynchronized), ("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066", TimeNotSynchronized), ("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50", TimeNotSynchronized), ("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 101864 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:42468 req_id = cfc14842-f9fd-48ca-a2a1-24dd5ff297ad response_code = 500 uri = /omicron-zones
9382025-01-10T21:55:13.314ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "cfc14842-f9fd-48ca-a2a1-24dd5ff297ad", "content-length": "124", "date": "Fri, 10 Jan 2025 21:55:13 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "cfc14842-f9fd-48ca-a2a1-24dd5ff297ad" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 86.783588ms sled_address = [fd00:1122:3344:101::1]:12345
9392025-01-10T21:55:13.402ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: e59b9a62-bb9f-4749-8a33-85254995194f (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 282e2202-986b-4d4d-a7ee-e57d0fc63594 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 6f2bb3b2-aa25-411c-817d-fd4c720ec487 (service), filesystem_pool: Some(ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 04fec8c4-c1f3-4609-8a2e-2fbb5b456603 (service), filesystem_pool: Some(ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 96f2491a-6c10-433d-a3d4-7d45935ab973, kind: Service { id: 04fec8c4-c1f3-4609-8a2e-2fbb5b456603 }, name: Name("ntp-04fec8c4-c1f3-4609-8a2e-2fbb5b456603"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 48])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 1b973bfa-6720-4fea-927a-5007c3c3fd48 (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } } } }, OmicronZoneConfig { id: 57b13ff2-7f23-49d7-93f9-9e02946b4c42 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9d57690c-8d29-49b6-a53d-5133c512e066 (service), filesystem_pool: Some(ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4a1ebfac-ae5e-4868-980a-299896e9fc32 (service), filesystem_pool: Some(ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External } } } }, OmicronZoneConfig { id: 80d005ab-2ce7-46d4-adc4-041a00673c50 (service), filesystem_pool: Some(ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
9402025-01-10T21:55:13.402ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9412025-01-10T21:55:13.402ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9422025-01-10T21:55:13.410ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9432025-01-10T21:55:13.410ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9442025-01-10T21:55:13.419ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9452025-01-10T21:55:13.419ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9462025-01-10T21:55:13.427ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9472025-01-10T21:55:13.427ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9482025-01-10T21:55:13.436ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9492025-01-10T21:55:13.436ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9502025-01-10T21:55:13.445ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9512025-01-10T21:55:13.445ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9522025-01-10T21:55:13.454ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9532025-01-10T21:55:13.454ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9542025-01-10T21:55:13.462ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9552025-01-10T21:55:13.462ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9562025-01-10T21:55:13.472ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9572025-01-10T21:55:13.472ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-zones.json file = common/src/ledger.rs:177
9582025-01-10T21:55:13.472ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-zones.json file = common/src/ledger.rs:177
9592025-01-10T21:55:13.472ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9602025-01-10T21:55:13.502ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
9612025-01-10T21:55:13.502ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
9622025-01-10T21:55:13.502ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
9632025-01-10T21:55:13.502ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
9642025-01-10T21:55:13.502ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
9652025-01-10T21:55:13.502ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066", TimeNotSynchronized), ("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48", TimeNotSynchronized), ("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42", TimeNotSynchronized), ("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32", TimeNotSynchronized), ("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 100293 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:42468 req_id = b5d05142-603c-4431-b640-6a9051fa3fae response_code = 500 uri = /omicron-zones
9662025-01-10T21:55:13.502ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "b5d05142-603c-4431-b640-6a9051fa3fae", "content-length": "124", "date": "Fri, 10 Jan 2025 21:55:13 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "b5d05142-603c-4431-b640-6a9051fa3fae" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 168.60104ms sled_address = [fd00:1122:3344:101::1]:12345
9672025-01-10T21:55:13.672ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: e59b9a62-bb9f-4749-8a33-85254995194f (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 282e2202-986b-4d4d-a7ee-e57d0fc63594 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 6f2bb3b2-aa25-411c-817d-fd4c720ec487 (service), filesystem_pool: Some(ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 04fec8c4-c1f3-4609-8a2e-2fbb5b456603 (service), filesystem_pool: Some(ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 96f2491a-6c10-433d-a3d4-7d45935ab973, kind: Service { id: 04fec8c4-c1f3-4609-8a2e-2fbb5b456603 }, name: Name("ntp-04fec8c4-c1f3-4609-8a2e-2fbb5b456603"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 48])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 1b973bfa-6720-4fea-927a-5007c3c3fd48 (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } } } }, OmicronZoneConfig { id: 57b13ff2-7f23-49d7-93f9-9e02946b4c42 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9d57690c-8d29-49b6-a53d-5133c512e066 (service), filesystem_pool: Some(ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4a1ebfac-ae5e-4868-980a-299896e9fc32 (service), filesystem_pool: Some(ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External } } } }, OmicronZoneConfig { id: 80d005ab-2ce7-46d4-adc4-041a00673c50 (service), filesystem_pool: Some(ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
9682025-01-10T21:55:13.672ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9692025-01-10T21:55:13.672ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9702025-01-10T21:55:13.681ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9712025-01-10T21:55:13.681ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9722025-01-10T21:55:13.690ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9732025-01-10T21:55:13.690ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9742025-01-10T21:55:13.699ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9752025-01-10T21:55:13.699ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9762025-01-10T21:55:13.709ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9772025-01-10T21:55:13.709ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9782025-01-10T21:55:13.718ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9792025-01-10T21:55:13.718ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9802025-01-10T21:55:13.726ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9812025-01-10T21:55:13.726ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9822025-01-10T21:55:13.735ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9832025-01-10T21:55:13.735ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9842025-01-10T21:55:13.743ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9852025-01-10T21:55:13.743ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-zones.json file = common/src/ledger.rs:177
9862025-01-10T21:55:13.743ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-zones.json file = common/src/ledger.rs:177
9872025-01-10T21:55:13.744ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9882025-01-10T21:55:13.774ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
9892025-01-10T21:55:13.774ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
9902025-01-10T21:55:13.774ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
9912025-01-10T21:55:13.774ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
9922025-01-10T21:55:13.774ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
9932025-01-10T21:55:13.774ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42", TimeNotSynchronized), ("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066", TimeNotSynchronized), ("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32", TimeNotSynchronized), ("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48", TimeNotSynchronized), ("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 101768 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:42468 req_id = 37e5d621-977f-43eb-b475-67ffd4038d87 response_code = 500 uri = /omicron-zones
9942025-01-10T21:55:13.774ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "37e5d621-977f-43eb-b475-67ffd4038d87", "content-length": "124", "date": "Fri, 10 Jan 2025 21:55:13 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "37e5d621-977f-43eb-b475-67ffd4038d87" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 140.297738ms sled_address = [fd00:1122:3344:101::1]:12345
9952025-01-10T21:55:13.915ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: e59b9a62-bb9f-4749-8a33-85254995194f (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 282e2202-986b-4d4d-a7ee-e57d0fc63594 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 6f2bb3b2-aa25-411c-817d-fd4c720ec487 (service), filesystem_pool: Some(ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 04fec8c4-c1f3-4609-8a2e-2fbb5b456603 (service), filesystem_pool: Some(ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 96f2491a-6c10-433d-a3d4-7d45935ab973, kind: Service { id: 04fec8c4-c1f3-4609-8a2e-2fbb5b456603 }, name: Name("ntp-04fec8c4-c1f3-4609-8a2e-2fbb5b456603"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 48])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 1b973bfa-6720-4fea-927a-5007c3c3fd48 (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } } } }, OmicronZoneConfig { id: 57b13ff2-7f23-49d7-93f9-9e02946b4c42 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9d57690c-8d29-49b6-a53d-5133c512e066 (service), filesystem_pool: Some(ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4a1ebfac-ae5e-4868-980a-299896e9fc32 (service), filesystem_pool: Some(ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External } } } }, OmicronZoneConfig { id: 80d005ab-2ce7-46d4-adc4-041a00673c50 (service), filesystem_pool: Some(ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
9962025-01-10T21:55:13.916ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9972025-01-10T21:55:13.916ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9982025-01-10T21:55:13.925ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9992025-01-10T21:55:13.925ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10002025-01-10T21:55:13.933ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10012025-01-10T21:55:13.933ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10022025-01-10T21:55:13.942ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10032025-01-10T21:55:13.942ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10042025-01-10T21:55:13.950ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10052025-01-10T21:55:13.950ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10062025-01-10T21:55:13.959ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10072025-01-10T21:55:13.959ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10082025-01-10T21:55:13.969ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10092025-01-10T21:55:13.969ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10102025-01-10T21:55:13.978ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10112025-01-10T21:55:13.978ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10122025-01-10T21:55:13.986ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10132025-01-10T21:55:13.986ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-zones.json file = common/src/ledger.rs:177
10142025-01-10T21:55:13.986ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-zones.json file = common/src/ledger.rs:177
10152025-01-10T21:55:13.987ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10162025-01-10T21:55:14.017ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
10172025-01-10T21:55:14.017ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
10182025-01-10T21:55:14.017ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
10192025-01-10T21:55:14.017ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
10202025-01-10T21:55:14.017ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3340 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
10212025-01-10T21:55:14.017ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42", TimeNotSynchronized), ("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066", TimeNotSynchronized), ("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48", TimeNotSynchronized), ("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32", TimeNotSynchronized), ("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 101871 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:42468 req_id = 9cc6989f-94ce-422f-96a8-069c143d43aa response_code = 500 uri = /omicron-zones
10222025-01-10T21:55:14.017ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "9cc6989f-94ce-422f-96a8-069c143d43aa", "content-length": "124", "date": "Fri, 10 Jan 2025 21:55:13 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "9cc6989f-94ce-422f-96a8-069c143d43aa" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 253.644147ms sled_address = [fd00:1122:3344:101::1]:12345
10232025-01-10T21:55:14.271ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: e59b9a62-bb9f-4749-8a33-85254995194f (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 282e2202-986b-4d4d-a7ee-e57d0fc63594 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 6f2bb3b2-aa25-411c-817d-fd4c720ec487 (service), filesystem_pool: Some(ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 04fec8c4-c1f3-4609-8a2e-2fbb5b456603 (service), filesystem_pool: Some(ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 96f2491a-6c10-433d-a3d4-7d45935ab973, kind: Service { id: 04fec8c4-c1f3-4609-8a2e-2fbb5b456603 }, name: Name("ntp-04fec8c4-c1f3-4609-8a2e-2fbb5b456603"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 48])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 1b973bfa-6720-4fea-927a-5007c3c3fd48 (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } } } }, OmicronZoneConfig { id: 57b13ff2-7f23-49d7-93f9-9e02946b4c42 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9d57690c-8d29-49b6-a53d-5133c512e066 (service), filesystem_pool: Some(ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4a1ebfac-ae5e-4868-980a-299896e9fc32 (service), filesystem_pool: Some(ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External } } } }, OmicronZoneConfig { id: 80d005ab-2ce7-46d4-adc4-041a00673c50 (service), filesystem_pool: Some(ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
10242025-01-10T21:55:14.272ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10252025-01-10T21:55:14.272ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10262025-01-10T21:55:14.280ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10272025-01-10T21:55:14.280ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10282025-01-10T21:55:14.289ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10292025-01-10T21:55:14.289ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10302025-01-10T21:55:14.298ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10312025-01-10T21:55:14.298ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10322025-01-10T21:55:14.307ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10332025-01-10T21:55:14.307ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10342025-01-10T21:55:14.315ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10352025-01-10T21:55:14.316ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10362025-01-10T21:55:14.323ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10372025-01-10T21:55:14.323ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10382025-01-10T21:55:14.332ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10392025-01-10T21:55:14.332ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10402025-01-10T21:55:14.341ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10412025-01-10T21:55:14.341ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-zones.json file = common/src/ledger.rs:177
10422025-01-10T21:55:14.341ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-zones.json file = common/src/ledger.rs:177
10432025-01-10T21:55:14.341ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10442025-01-10T21:55:14.422ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10452025-01-10T21:55:14.422ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10462025-01-10T21:55:14.422ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10472025-01-10T21:55:14.422ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10482025-01-10T21:55:14.422ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10492025-01-10T21:55:14.510ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32 file = illumos-utils/src/zone.rs:325
10502025-01-10T21:55:14.515ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42 file = illumos-utils/src/zone.rs:325
10512025-01-10T21:55:14.519ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066 file = illumos-utils/src/zone.rs:325
10522025-01-10T21:55:14.523ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50 file = illumos-utils/src/zone.rs:325
10532025-01-10T21:55:14.527ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48 file = illumos-utils/src/zone.rs:325
10542025-01-10T21:55:14.532ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32 file = illumos-utils/src/zone.rs:363
10552025-01-10T21:55:14.537ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42 file = illumos-utils/src/zone.rs:363
10562025-01-10T21:55:14.542ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066 file = illumos-utils/src/zone.rs:363
10572025-01-10T21:55:14.547ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50 file = illumos-utils/src/zone.rs:363
10582025-01-10T21:55:14.551ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48 file = illumos-utils/src/zone.rs:363
10592025-01-10T21:55:17.692ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService7'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='9d57690c-8d29-49b6-a53d-5133c512e066'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::5]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10602025-01-10T21:55:17.692ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
10612025-01-10T21:55:17.701ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService6'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::4'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='57b13ff2-7f23-49d7-93f9-9e02946b4c42'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::4]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10622025-01-10T21:55:17.701ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
10632025-01-10T21:55:17.751ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService9'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::3'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='1b973bfa-6720-4fea-927a-5007c3c3fd48'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::3]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10642025-01-10T21:55:17.751ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
10652025-01-10T21:55:17.758ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService8'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='80d005ab-2ce7-46d4-adc4-041a00673c50'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::7]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10662025-01-10T21:55:17.758ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
10672025-01-10T21:55:17.795ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService5'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='4a1ebfac-ae5e-4868-980a-299896e9fc32'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::6]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10682025-01-10T21:55:17.795ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
10692025-01-10T21:55:17.840ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42") failed: Property not found. retry in 39.438759ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
10702025-01-10T21:55:17.849ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066") failed: Property not found. retry in 35.367986ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
10712025-01-10T21:55:17.879ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48") failed: Property not found. retry in 57.251489ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
10722025-01-10T21:55:17.888ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42") failed: Property not found. retry in 127.111256ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
10732025-01-10T21:55:17.895ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066") failed: Property not found. retry in 104.385335ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
10742025-01-10T21:55:17.902ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50") failed: Property not found. retry in 48.376887ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
10752025-01-10T21:55:18.087ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50") failed: Property not found. retry in 121.936374ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
10762025-01-10T21:55:18.189ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48") failed: Property not found. retry in 138.28874ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
10772025-01-10T21:55:18.587ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42") failed: Property not found. retry in 162.648441ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
10782025-01-10T21:55:18.630ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066") failed: Property not found. retry in 126.401235ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
10792025-01-10T21:55:18.659ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50") failed: Property not found. retry in 117.045838ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
10802025-01-10T21:55:18.703ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48") failed: Property not found. retry in 233.780776ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
10812025-01-10T21:55:18.928ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32") failed: Property not found. retry in 41.412625ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
10822025-01-10T21:55:19.018ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42") failed: Property not found. retry in 549.896626ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
10832025-01-10T21:55:19.038ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066") failed: Property not found. retry in 306.05864ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
10842025-01-10T21:55:19.058ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50") failed: Property not found. retry in 575.895298ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
10852025-01-10T21:55:19.077ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48") failed: Property not found. retry in 477.119047ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
10862025-01-10T21:55:19.093ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32") failed: Property not found. retry in 99.003551ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
10872025-01-10T21:55:19.212ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32") failed: Property not found. retry in 228.491902ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
10882025-01-10T21:55:19.224ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:44713
10892025-01-10T21:55:19.225ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 128 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:44713 req_id = 197183ef-c322-441a-9260-28e287be7129 response_code = 200 uri = /baseboard
10902025-01-10T21:55:19.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066") failed: Property not found. retry in 488.899102ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
10912025-01-10T21:55:19.452ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32") failed: Property not found. retry in 403.774561ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
10922025-01-10T21:55:19.564ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48") failed: Property not found. retry in 628.601508ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
10932025-01-10T21:55:19.576ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42") failed: Property not found. retry in 1.187359875s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
10942025-01-10T21:55:19.644ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50") failed: Property not found. retry in 956.215419ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
10952025-01-10T21:55:19.855ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066") failed: Property not found. retry in 787.883685ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
10962025-01-10T21:55:19.866ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32") failed: Property not found. retry in 1.038954176s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
10972025-01-10T21:55:20.201ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48") failed: Property not found. retry in 1.112256041s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
10982025-01-10T21:55:20.609ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50") failed: Property not found. retry in 1.46942967s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
10992025-01-10T21:55:20.652ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066") failed: Property not found. retry in 970.25051ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
11002025-01-10T21:55:20.772ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42") failed: Property not found. retry in 953.905195ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
11012025-01-10T21:55:20.911ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32") failed: Property not found. retry in 1.113435223s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
11022025-01-10T21:55:21.320ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48") failed: Property not found. retry in 621.693132ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
11032025-01-10T21:55:21.628ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066") failed: Property not found. retry in 1.080381969s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
11042025-01-10T21:55:21.732ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42") failed: Property not found. retry in 840.081407ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
11052025-01-10T21:55:21.949ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48") failed: Property not found. retry in 1.415320669s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
11062025-01-10T21:55:22.030ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32") failed: Property not found. retry in 1.240285758s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
11072025-01-10T21:55:22.084ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50") failed: Property not found. retry in 1.331514082s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
11082025-01-10T21:55:22.579ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42") failed: Property not found. retry in 1.295095049s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
11092025-01-10T21:55:22.714ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066") failed: Property not found. retry in 1.234514738s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
11102025-01-10T21:55:23.276ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32") failed: Property not found. retry in 558.254544ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
11112025-01-10T21:55:23.370ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48") failed: Property not found. retry in 1.267744615s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
11122025-01-10T21:55:23.420ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50") failed: Property not found. retry in 1.050456599s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
11132025-01-10T21:55:23.841ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32") failed: Property not found. retry in 1.267749529s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
11142025-01-10T21:55:23.879ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42") failed: Property not found. retry in 863.218047ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
11152025-01-10T21:55:23.954ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066") failed: Property not found. retry in 1.481338993s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
11162025-01-10T21:55:24.477ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50") failed: Property not found. retry in 616.156852ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
11172025-01-10T21:55:24.644ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48") failed: Property not found. retry in 1.333330597s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
11182025-01-10T21:55:24.748ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42") failed: Property not found. retry in 538.549515ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
11192025-01-10T21:55:25.100ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50") failed: Property not found. retry in 1.034809131s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
11202025-01-10T21:55:25.114ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32") failed: Property not found. retry in 988.879393ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
11212025-01-10T21:55:25.294ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42") failed: Property not found. retry in 775.486443ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
11222025-01-10T21:55:25.442ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066") failed: Property not found. retry in 1.440726748s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
11232025-01-10T21:55:25.984ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48") failed: Property not found. retry in 973.938028ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
11242025-01-10T21:55:26.078ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42") failed: Property not found. retry in 779.593895ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
11252025-01-10T21:55:26.109ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32") failed: Property not found. retry in 740.076002ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
11262025-01-10T21:55:26.140ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50") failed: Property not found. retry in 607.910835ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
11272025-01-10T21:55:26.757ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50") failed: Property not found. retry in 1.478039195s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
11282025-01-10T21:55:26.861ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32") failed: Property not found. retry in 767.544226ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
11292025-01-10T21:55:26.870ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42") failed: Property not found. retry in 520.612648ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
11302025-01-10T21:55:26.890ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066") failed: Property not found. retry in 1.253378849s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
11312025-01-10T21:55:26.967ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48") failed: Property not found. retry in 1.345247408s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
11322025-01-10T21:55:27.399ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42") failed: Property not found. retry in 1.302499594s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
11332025-01-10T21:55:27.637ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32") failed: Property not found. retry in 614.65073ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
11342025-01-10T21:55:28.151ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066") failed: Property not found. retry in 947.844759ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
11352025-01-10T21:55:28.241ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50") failed: Property not found. retry in 693.816136ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
11362025-01-10T21:55:28.260ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32") failed: Property not found. retry in 1.175261117s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
11372025-01-10T21:55:28.320ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48") failed: Property not found. retry in 570.02046ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
11382025-01-10T21:55:28.711ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42") failed: Property not found. retry in 1.219567727s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
11392025-01-10T21:55:28.899ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48") failed: Property not found. retry in 1.078079064s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
11402025-01-10T21:55:28.945ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50") failed: Property not found. retry in 1.356125979s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
11412025-01-10T21:55:29.108ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066") failed: Property not found. retry in 1.265241259s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
11422025-01-10T21:55:29.447ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32") failed: Property not found. retry in 1.276462999s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
11432025-01-10T21:55:29.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42") failed: Property not found. retry in 1.38544374s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
11442025-01-10T21:55:29.987ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48") failed: Property not found. retry in 1.028252506s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
11452025-01-10T21:55:30.313ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50") failed: Property not found. retry in 1.371281665s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
11462025-01-10T21:55:30.382ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066") failed: Property not found. retry in 1.034058591s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
11472025-01-10T21:55:31.707ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_cockroachdb_4a1ebfac-ae5e-4868-980a-299896e9fc32
11482025-01-10T21:55:31.708ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_cockroachdb_57b13ff2-7f23-49d7-93f9-9e02946b4c42
11492025-01-10T21:55:31.708ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_cockroachdb_9d57690c-8d29-49b6-a53d-5133c512e066
11502025-01-10T21:55:31.708ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_cockroachdb_80d005ab-2ce7-46d4-adc4-041a00673c50
11512025-01-10T21:55:31.708ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48
11522025-01-10T21:55:31.708ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
11532025-01-10T21:55:31.708ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/10481327-13d1-4548-950f-130d063e2690/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
11542025-01-10T21:55:31.708ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 17472113 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:42468 req_id = 3c0b0454-6c55-49be-8867-3cb719a330cc response_code = 204 uri = /omicron-zones
11552025-01-10T21:55:31.743ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60891
11562025-01-10T21:55:31.774ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 30518 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:60891 req_id = 5a7f7cff-f66e-4052-b416-c3306e116ce3 response_code = 500 uri = /cockroachdb
11572025-01-10T21:55:31.774ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "5a7f7cff-f66e-4052-b416-c3306e116ce3", "content-length": "124", "date": "Fri, 10 Jan 2025 21:55:31 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "5a7f7cff-f66e-4052-b416-c3306e116ce3" } file = sled-agent/src/rack_setup/service.rs:1158 retry_after = 91.064929ms
11582025-01-10T21:55:31.893ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 26664 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:60891 req_id = e1d500d4-41b5-4045-a9ce-136a4e8eb6a5 response_code = 500 uri = /cockroachdb
11592025-01-10T21:55:31.893ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "e1d500d4-41b5-4045-a9ce-136a4e8eb6a5", "content-length": "124", "date": "Fri, 10 Jan 2025 21:55:31 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "e1d500d4-41b5-4045-a9ce-136a4e8eb6a5" } file = sled-agent/src/rack_setup/service.rs:1158 retry_after = 129.853185ms
11602025-01-10T21:55:32.052ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 28683 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:60891 req_id = 54310baf-da5f-4e67-9b34-76f7ffff850b response_code = 500 uri = /cockroachdb
11612025-01-10T21:55:32.052ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "54310baf-da5f-4e67-9b34-76f7ffff850b", "content-length": "124", "date": "Fri, 10 Jan 2025 21:55:31 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "54310baf-da5f-4e67-9b34-76f7ffff850b" } file = sled-agent/src/rack_setup/service.rs:1158 retry_after = 115.226785ms
11622025-01-10T21:55:32.196ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 27086 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:60891 req_id = 5d184812-2c9b-4962-bc8a-c0eab95644ae response_code = 500 uri = /cockroachdb
11632025-01-10T21:55:32.196ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "5d184812-2c9b-4962-bc8a-c0eab95644ae", "content-length": "124", "date": "Fri, 10 Jan 2025 21:55:31 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "5d184812-2c9b-4962-bc8a-c0eab95644ae" } file = sled-agent/src/rack_setup/service.rs:1158 retry_after = 163.795373ms
11642025-01-10T21:55:32.211ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 242.954947ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
11652025-01-10T21:55:32.387ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService9/omicron6 in oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_1b973bfa-6720-4fea-927a-5007c3c3fd48 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService9/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 26724 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:60891 req_id = 245be51b-05e0-4aa5-9f1c-03ad082891d1 response_code = 500 uri = /cockroachdb
11662025-01-10T21:55:32.388ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "245be51b-05e0-4aa5-9f1c-03ad082891d1", "content-length": "124", "date": "Fri, 10 Jan 2025 21:55:31 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "245be51b-05e0-4aa5-9f1c-03ad082891d1" } file = sled-agent/src/rack_setup/service.rs:1158 retry_after = 271.862894ms
11672025-01-10T21:55:32.695ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::3]:32221 file = sled-agent/src/services.rs:3769
11682025-01-10T21:55:49.862ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3787
11692025-01-10T21:55:54.634ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:3810
11702025-01-10T21:55:54.635ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 21974126 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:60891 req_id = 5a2e759e-bd1d-4833-856f-d4c1dfe52de7 response_code = 204 uri = /cockroachdb
11712025-01-10T21:55:54.641ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:37658
11722025-01-10T21:55:54.643ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 554 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:37658 req_id = f63bd3aa-bcf9-4aa8-9bb4-38e9c8b86cd2 response_code = 200 uri = /baseboard
11732025-01-10T21:55:54.656ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 569.262859ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
11742025-01-10T21:55:54.691ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: e59b9a62-bb9f-4749-8a33-85254995194f (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 282e2202-986b-4d4d-a7ee-e57d0fc63594 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 6f2bb3b2-aa25-411c-817d-fd4c720ec487 (service), filesystem_pool: Some(ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 04fec8c4-c1f3-4609-8a2e-2fbb5b456603 (service), filesystem_pool: Some(ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 96f2491a-6c10-433d-a3d4-7d45935ab973, kind: Service { id: 04fec8c4-c1f3-4609-8a2e-2fbb5b456603 }, name: Name("ntp-04fec8c4-c1f3-4609-8a2e-2fbb5b456603"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 179, 48])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 1b973bfa-6720-4fea-927a-5007c3c3fd48 (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } } } }, OmicronZoneConfig { id: 57b13ff2-7f23-49d7-93f9-9e02946b4c42 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 9d57690c-8d29-49b6-a53d-5133c512e066 (service), filesystem_pool: Some(ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } } } }, OmicronZoneConfig { id: 4a1ebfac-ae5e-4868-980a-299896e9fc32 (service), filesystem_pool: Some(ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External } } } }, OmicronZoneConfig { id: 80d005ab-2ce7-46d4-adc4-041a00673c50 (service), filesystem_pool: Some(ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External } } } }, OmicronZoneConfig { id: bce93894-738c-4596-b983-281da931e977 (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.2.164:53, nic: NetworkInterface { id: 6f22ade2-da2d-4cbc-9141-adcc3b335aed, kind: Service { id: bce93894-738c-4596-b983-281da931e977 }, name: Name("external-dns-bce93894-738c-4596-b983-281da931e977"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 88])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: f44690fa-03a0-42b0-bb97-26c3d20aac30 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.2.165:53, nic: NetworkInterface { id: 6decc476-dbff-4cb1-a909-d9d4408adaab, kind: Service { id: f44690fa-03a0-42b0-bb97-26c3d20aac30 }, name: Name("external-dns-f44690fa-03a0-42b0-bb97-26c3d20aac30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 143])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: 4106fd29-fc2d-4538-b321-136cbc8ea9d2 (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.2.166, nic: NetworkInterface { id: 8448bae1-35b5-443a-8e61-6f2848603054, kind: Service { id: 4106fd29-fc2d-4538-b321-136cbc8ea9d2 }, name: Name("nexus-4106fd29-fc2d-4538-b321-136cbc8ea9d2"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 27])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 19558293-ba4c-462e-8746-8d928c3a9612 (service), filesystem_pool: Some(ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.2.167, nic: NetworkInterface { id: 9676df2d-b2c5-40f4-ae20-30ad4de6c8e2, kind: Service { id: 19558293-ba4c-462e-8746-8d928c3a9612 }, name: Name("nexus-19558293-ba4c-462e-8746-8d928c3a9612"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 42])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: b507bc15-7e7b-4d56-a806-1da438b76052 (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.2.168, nic: NetworkInterface { id: bf22f522-6412-4896-90b5-9bc8aab0192d, kind: Service { id: b507bc15-7e7b-4d56-a806-1da438b76052 }, name: Name("nexus-b507bc15-7e7b-4d56-a806-1da438b76052"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 137])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 17551ca7-3205-43ba-9a16-734e9902cf2f (service), filesystem_pool: Some(ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }), zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: e1609ebd-d07d-4215-85da-1ba67a8787ee (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } } } }, OmicronZoneConfig { id: 242b32ff-c26f-43e2-9af7-3c5313c43047 (service), filesystem_pool: Some(ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f (service), filesystem_pool: Some(ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: d7055447-403c-4747-bc61-71c3d82805fc (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: 4b7768ac-7d53-4548-864f-2d97118bfb78 (service), filesystem_pool: Some(ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External } } } }, OmicronZoneConfig { id: 955aa9c6-e944-4ccc-9586-f727632ff66a (service), filesystem_pool: Some(ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External } } } }, OmicronZoneConfig { id: 6e9711ba-8ebd-4b53-ad66-96b22d084d7b (service), filesystem_pool: Some(ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External } } } }, OmicronZoneConfig { id: fba7a706-948d-4c47-8257-491e17a3a4f2 (service), filesystem_pool: Some(ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External } } } }, OmicronZoneConfig { id: 3c40e645-d883-49f8-9f90-44e515e5096d (service), filesystem_pool: Some(ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External } } } }, OmicronZoneConfig { id: ef867d0b-1af9-4769-a1f4-6030cbcf7090 (service), filesystem_pool: Some(ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External } } } }, OmicronZoneConfig { id: f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd (service), filesystem_pool: Some(ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External } } } }, OmicronZoneConfig { id: f6578145-7a5a-4961-a28a-6e01465004d6 (service), filesystem_pool: Some(ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External } } } }, OmicronZoneConfig { id: 8774e52d-c771-41cc-a8ab-79387985e935 (service), filesystem_pool: Some(ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
11752025-01-10T21:55:54.692ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:61302
11762025-01-10T21:55:54.695ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
11772025-01-10T21:55:54.695ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: InternalDns }, responder: ... }
11782025-01-10T21:55:54.708ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
11792025-01-10T21:55:54.708ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: InternalDns }, responder: ... }
11802025-01-10T21:55:54.718ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
11812025-01-10T21:55:54.718ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: InternalDns }, responder: ... }
11822025-01-10T21:55:54.729ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
11832025-01-10T21:55:54.729ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Cockroach }, responder: ... }
11842025-01-10T21:55:54.739ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
11852025-01-10T21:55:54.739ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Cockroach }, responder: ... }
11862025-01-10T21:55:54.748ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
11872025-01-10T21:55:54.748ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Cockroach }, responder: ... }
11882025-01-10T21:55:54.758ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
11892025-01-10T21:55:54.758ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Cockroach }, responder: ... }
11902025-01-10T21:55:54.768ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
11912025-01-10T21:55:54.768ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Cockroach }, responder: ... }
11922025-01-10T21:55:54.777ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
11932025-01-10T21:55:54.777ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
11942025-01-10T21:55:54.787ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
11952025-01-10T21:55:54.787ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
11962025-01-10T21:55:54.799ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Clickhouse }, responder: ... }) file = sled-storage/src/manager.rs:607
11972025-01-10T21:55:54.799ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Clickhouse }, responder: ... }
11982025-01-10T21:55:54.810ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
11992025-01-10T21:55:54.810ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, kind: Crucible }, responder: ... }
12002025-01-10T21:55:54.821ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12012025-01-10T21:55:54.821ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, kind: Crucible }, responder: ... }
12022025-01-10T21:55:54.831ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12032025-01-10T21:55:54.831ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, kind: Crucible }, responder: ... }
12042025-01-10T21:55:54.841ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12052025-01-10T21:55:54.841ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, kind: Crucible }, responder: ... }
12062025-01-10T21:55:54.851ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12072025-01-10T21:55:54.851ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, kind: Crucible }, responder: ... }
12082025-01-10T21:55:54.861ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12092025-01-10T21:55:54.861ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External }, kind: Crucible }, responder: ... }
12102025-01-10T21:55:54.871ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12112025-01-10T21:55:54.871ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }, kind: Crucible }, responder: ... }
12122025-01-10T21:55:54.881ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12132025-01-10T21:55:54.881ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }, kind: Crucible }, responder: ... }
12142025-01-10T21:55:54.890ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12152025-01-10T21:55:54.890ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, kind: Crucible }, responder: ... }
12162025-01-10T21:55:54.900ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12172025-01-10T21:55:54.901ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-zones.json file = common/src/ledger.rs:177
12182025-01-10T21:55:54.901ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-zones.json file = common/src/ledger.rs:177
12192025-01-10T21:55:54.902ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12202025-01-10T21:55:55.020ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12212025-01-10T21:55:55.021ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12222025-01-10T21:55:55.045ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12232025-01-10T21:55:55.046ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12242025-01-10T21:55:55.073ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12252025-01-10T21:55:55.073ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12262025-01-10T21:55:55.073ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12272025-01-10T21:55:55.073ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12282025-01-10T21:55:55.085ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12292025-01-10T21:55:55.098ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12302025-01-10T21:55:55.098ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12312025-01-10T21:55:55.283ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12322025-01-10T21:55:55.284ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12332025-01-10T21:55:55.284ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12342025-01-10T21:55:55.284ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12352025-01-10T21:55:55.284ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12362025-01-10T21:55:55.310ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12372025-01-10T21:55:55.310ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12382025-01-10T21:55:55.310ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12392025-01-10T21:55:55.310ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd file = illumos-utils/src/zone.rs:325
12402025-01-10T21:55:55.316ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6 file = illumos-utils/src/zone.rs:325
12412025-01-10T21:55:55.321ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047 file = illumos-utils/src/zone.rs:325
12422025-01-10T21:55:55.327ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee file = illumos-utils/src/zone.rs:325
12432025-01-10T21:55:55.332ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b file = illumos-utils/src/zone.rs:325
12442025-01-10T21:55:55.336ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f file = illumos-utils/src/zone.rs:325
12452025-01-10T21:55:55.341ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78 file = illumos-utils/src/zone.rs:325
12462025-01-10T21:55:55.475ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd file = illumos-utils/src/zone.rs:363
12472025-01-10T21:55:55.480ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6 file = illumos-utils/src/zone.rs:363
12482025-01-10T21:55:55.484ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047 file = illumos-utils/src/zone.rs:363
12492025-01-10T21:55:55.489ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee file = illumos-utils/src/zone.rs:363
12502025-01-10T21:55:55.493ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b file = illumos-utils/src/zone.rs:363
12512025-01-10T21:55:55.498ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f file = illumos-utils/src/zone.rs:363
12522025-01-10T21:55:55.503ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78 file = illumos-utils/src/zone.rs:363
12532025-01-10T21:55:55.508ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc file = illumos-utils/src/zone.rs:325
12542025-01-10T21:55:55.513ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935 file = illumos-utils/src/zone.rs:325
12552025-01-10T21:55:55.517ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090 file = illumos-utils/src/zone.rs:325
12562025-01-10T21:55:55.522ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d file = illumos-utils/src/zone.rs:325
12572025-01-10T21:55:55.527ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a file = illumos-utils/src/zone.rs:325
12582025-01-10T21:55:55.532ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f file = illumos-utils/src/zone.rs:325
12592025-01-10T21:55:55.537ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2 file = illumos-utils/src/zone.rs:325
12602025-01-10T21:55:55.602ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12612025-01-10T21:55:55.603ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc file = illumos-utils/src/zone.rs:363
12622025-01-10T21:55:55.616ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935 file = illumos-utils/src/zone.rs:363
12632025-01-10T21:55:55.634ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090 file = illumos-utils/src/zone.rs:363
12642025-01-10T21:55:55.647ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d file = illumos-utils/src/zone.rs:363
12652025-01-10T21:55:55.660ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a file = illumos-utils/src/zone.rs:363
12662025-01-10T21:55:55.667ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f file = illumos-utils/src/zone.rs:363
12672025-01-10T21:55:55.673ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2 file = illumos-utils/src/zone.rs:363
12682025-01-10T21:55:55.733ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12692025-01-10T21:55:55.791ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12702025-01-10T21:55:55.848ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12712025-01-10T21:55:55.850ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
12722025-01-10T21:55:55.850ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
12732025-01-10T21:55:55.906ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte1", ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 241, 137]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
12742025-01-10T21:55:55.906ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1275 zone_type = nexus
12752025-01-10T21:55:55.973ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12762025-01-10T21:55:55.975ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
12772025-01-10T21:55:55.975ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
12782025-01-10T21:55:56.035ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte2", ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 221, 27]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
12792025-01-10T21:55:56.035ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1275 zone_type = nexus
12802025-01-10T21:55:56.037ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
12812025-01-10T21:55:56.037ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
12822025-01-10T21:55:56.095ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte3", ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 166, 88]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
12832025-01-10T21:55:56.095ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1275 zone_type = external_dns
12842025-01-10T21:55:56.098ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
12852025-01-10T21:55:56.098ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
12862025-01-10T21:55:56.165ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte4", ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 175, 143]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
12872025-01-10T21:55:56.165ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1275 zone_type = external_dns
12882025-01-10T21:55:56.165ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:83
12892025-01-10T21:55:56.165ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:90
12902025-01-10T21:55:56.166ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:83
12912025-01-10T21:55:56.166ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:90
12922025-01-10T21:55:56.168ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
12932025-01-10T21:55:56.168ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
12942025-01-10T21:55:56.225ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte5", ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 160, 42]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
12952025-01-10T21:55:56.225ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1275 zone_type = nexus
12962025-01-10T21:55:56.226ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:118
12972025-01-10T21:55:56.227ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:83
12982025-01-10T21:55:56.227ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:90
12992025-01-10T21:55:56.227ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13002025-01-10T21:55:56.227ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:118
13012025-01-10T21:55:56.229ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13022025-01-10T21:55:56.332ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:118
13032025-01-10T21:55:56.441ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13042025-01-10T21:55:56.441ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:83
13052025-01-10T21:55:56.441ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:90
13062025-01-10T21:55:56.443ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052 file = illumos-utils/src/zone.rs:325
13072025-01-10T21:55:56.569ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2 file = illumos-utils/src/zone.rs:325
13082025-01-10T21:55:56.614ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:118
13092025-01-10T21:55:56.622ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13102025-01-10T21:55:56.622ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:83
13112025-01-10T21:55:56.622ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:90
13122025-01-10T21:55:56.623ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052 file = illumos-utils/src/zone.rs:363
13132025-01-10T21:55:56.647ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_bce93894-738c-4596-b983-281da931e977 file = illumos-utils/src/zone.rs:325
13142025-01-10T21:55:56.677ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2 file = illumos-utils/src/zone.rs:363
13152025-01-10T21:55:56.817ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:118
13162025-01-10T21:55:56.821ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13172025-01-10T21:55:56.821ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_bce93894-738c-4596-b983-281da931e977 file = illumos-utils/src/zone.rs:363
13182025-01-10T21:55:56.874ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30 file = illumos-utils/src/zone.rs:325
13192025-01-10T21:55:56.898ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612 file = illumos-utils/src/zone.rs:325
13202025-01-10T21:55:56.953ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30 file = illumos-utils/src/zone.rs:363
13212025-01-10T21:55:56.971ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612 file = illumos-utils/src/zone.rs:363
13222025-01-10T21:55:58.339ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService12'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::f'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::f'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13232025-01-10T21:55:58.339ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService15'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::10'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::10'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13242025-01-10T21:55:58.343ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
13252025-01-10T21:55:58.350ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
13262025-01-10T21:55:58.469ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService19'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::17'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_7b0e1b76-af4f-49bd-9848-611d8d8666ad/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::17'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='9d7c45b4-e036-472a-878e-a17dd4c37fda'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13272025-01-10T21:55:58.469ZINFOSledAgent (ServiceManager): Booting oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
13282025-01-10T21:55:58.477ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService20'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::16'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_ce5c629d-252d-4e12-9f9d-2087cb772d81/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::16'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='4a516c7e-52a1-4672-949e-318bd6a00010'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13292025-01-10T21:55:58.477ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService23'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::15'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_d27ec511-4e48-4e26-9677-295de0452e91/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::15'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='e13c421e-eaaf-41d2-90ab-4da1a16e788e'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13302025-01-10T21:55:58.477ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService10'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::18'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_619e5037-b196-43f6-b092-666b1b1387ff/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::18'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='d2859324-1990-49d4-8908-40d4320d364c'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13312025-01-10T21:55:58.477ZINFOSledAgent (ServiceManager): Booting oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
13322025-01-10T21:55:58.484ZINFOSledAgent (ServiceManager): Booting oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
13332025-01-10T21:55:58.491ZINFOSledAgent (ServiceManager): Booting oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
13342025-01-10T21:55:58.497ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService11'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::19'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_eecab578-d379-46e4-90f5-9dc11839dad5/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::19'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='33bd3f5b-d9e7-4bdc-b6a5-6c7e6ad7b06d'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13352025-01-10T21:55:58.498ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService14'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::14'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_5d7ad0a3-e47e-4369-b599-d02dbd3a8829/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::14'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='f2f329a8-e9ce-461f-9388-89b129dfde92'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13362025-01-10T21:55:58.498ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService18'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_784c4a6b-3804-44fa-afd4-c974bef9dd5d/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1a'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='1d1721f3-af4f-44df-a992-6f9f8d146529'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13372025-01-10T21:55:58.498ZINFOSledAgent (ServiceManager): Booting oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
13382025-01-10T21:55:58.503ZINFOSledAgent (ServiceManager): Booting oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
13392025-01-10T21:55:58.509ZINFOSledAgent (ServiceManager): Booting oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
13402025-01-10T21:55:58.701ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 30.890011ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
13412025-01-10T21:55:58.711ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 64.548972ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
13422025-01-10T21:55:58.721ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 61.162066ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
13432025-01-10T21:55:58.731ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 65.102127ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
13442025-01-10T21:55:58.741ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 44.843543ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
13452025-01-10T21:55:58.749ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 70.639901ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
13462025-01-10T21:55:58.749ZINFOSledAgent (ServiceManager): Profile for oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService22'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::d'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/oximeter"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="id" value='17551ca7-3205-43ba-9a16-734e9902cf2f'/> <propval type="astring" name="address" value='[fd00:1122:3344:101::d]:12223'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13472025-01-10T21:55:58.758ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 58.120768ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
13482025-01-10T21:55:58.759ZINFOSledAgent (ServiceManager): Booting oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
13492025-01-10T21:55:58.777ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 68.615081ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
13502025-01-10T21:55:58.802ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 107.907582ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
13512025-01-10T21:55:58.854ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 130.453773ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
13522025-01-10T21:55:58.888ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 97.754329ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
13532025-01-10T21:55:58.919ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 25.568028ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
13542025-01-10T21:55:58.956ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 149.805221ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
13552025-01-10T21:55:58.985ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 50.554944ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
13562025-01-10T21:55:59.018ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 57.668425ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
13572025-01-10T21:55:59.041ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 60.341206ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
13582025-01-10T21:55:59.066ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 103.13083ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
13592025-01-10T21:55:59.107ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 198.499933ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
13602025-01-10T21:55:59.142ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 155.170907ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
13612025-01-10T21:55:59.179ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 214.316881ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
13622025-01-10T21:55:59.205ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 104.048267ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
13632025-01-10T21:55:59.230ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 298.925682ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
13642025-01-10T21:55:59.252ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 140.073382ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
13652025-01-10T21:55:59.275ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 141.357404ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
13662025-01-10T21:55:59.351ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 162.459348ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
13672025-01-10T21:55:59.924ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 158.746984ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
13682025-01-10T21:56:00.041ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 398.444317ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
13692025-01-10T21:56:00.098ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 404.420253ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
13702025-01-10T21:56:00.135ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 493.876937ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
13712025-01-10T21:56:00.169ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 175.583714ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
13722025-01-10T21:56:00.204ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 575.359883ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
13732025-01-10T21:56:00.227ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 566.803291ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
13742025-01-10T21:56:00.247ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 48.187387ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
13752025-01-10T21:56:00.263ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 227.288128ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
13762025-01-10T21:56:00.281ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 494.438277ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
13772025-01-10T21:56:00.296ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 319.763185ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
13782025-01-10T21:56:00.311ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 78.020142ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
13792025-01-10T21:56:00.357ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 286.32059ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
13802025-01-10T21:56:00.405ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 260.015108ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
13812025-01-10T21:56:00.455ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 404.609233ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
13822025-01-10T21:56:00.508ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 893.144921ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
13832025-01-10T21:56:00.522ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 810.232901ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
13842025-01-10T21:56:00.614ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService16'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::12'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_73828a50-9586-442b-b526-668d3e1de808/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::12'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='037bbbe5-6030-4c6b-9255-1db512a3331e'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13852025-01-10T21:56:00.614ZINFOSledAgent (ServiceManager): Booting oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
13862025-01-10T21:56:00.637ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 713.289465ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
13872025-01-10T21:56:00.651ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 1.088385667s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
13882025-01-10T21:56:00.664ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 542.747568ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
13892025-01-10T21:56:00.680ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 268.014846ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
13902025-01-10T21:56:00.794ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 431.594367ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
13912025-01-10T21:56:00.809ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 657.572007ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
13922025-01-10T21:56:00.824ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 890.962328ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
13932025-01-10T21:56:00.876ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 1.09560138s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
13942025-01-10T21:56:00.895ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 68.361757ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
13952025-01-10T21:56:00.965ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 703.711329ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
13962025-01-10T21:56:00.977ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 116.02735ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
13972025-01-10T21:56:01.088ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_bce93894-738c-4596-b983-281da931e977: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService26'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::8'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte3'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::8]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.5:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13982025-01-10T21:56:01.090ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_bce93894-738c-4596-b983-281da931e977 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
13992025-01-10T21:56:01.110ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 273.506357ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
14002025-01-10T21:56:01.224ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 646.046202ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
14012025-01-10T21:56:01.242ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 939.481505ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
14022025-01-10T21:56:01.348ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 1.192623514s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
14032025-01-10T21:56:01.360ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 48.645346ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
14042025-01-10T21:56:01.375ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 1.412287206s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
14052025-01-10T21:56:01.402ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 360.084419ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
14062025-01-10T21:56:01.417ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 597.831559ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
14072025-01-10T21:56:01.428ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 60.028762ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
14082025-01-10T21:56:01.486ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 565.371626ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
14092025-01-10T21:56:01.501ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 193.538952ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
14102025-01-10T21:56:01.683ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 834.432814ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
14112025-01-10T21:56:01.708ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 291.376796ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
14122025-01-10T21:56:01.728ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 644.279648ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
14132025-01-10T21:56:01.752ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 1.047483735s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
14142025-01-10T21:56:01.772ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 1.070647747s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
14152025-01-10T21:56:01.882ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 1.258445909s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
14162025-01-10T21:56:01.986ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 1.108155648s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
14172025-01-10T21:56:02.013ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 1.160376505s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
14182025-01-10T21:56:02.027ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 1.021883934s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
14192025-01-10T21:56:02.064ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 1.217474026s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
14202025-01-10T21:56:02.195ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 729.466819ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
14212025-01-10T21:56:02.386ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 544.18715ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
14222025-01-10T21:56:02.532ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 1.286673278s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
14232025-01-10T21:56:02.553ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 809.970082ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
14242025-01-10T21:56:02.800ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 1.392563717s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
14252025-01-10T21:56:02.814ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 1.303258722s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
14262025-01-10T21:56:02.994ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 1.499684832s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
14272025-01-10T21:56:03.038ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 932.306175ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
14282025-01-10T21:56:03.095ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 1.406726248s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
14292025-01-10T21:56:03.156ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 793.811666ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
14302025-01-10T21:56:03.183ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 1.302880832s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
14312025-01-10T21:56:03.196ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 531.641195ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
14322025-01-10T21:56:03.214ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 506.308496ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
14332025-01-10T21:56:03.297ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 1.082158674s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
14342025-01-10T21:56:03.378ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 1.430379696s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
14352025-01-10T21:56:03.735ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 1.204831503s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
14362025-01-10T21:56:03.746ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 1.079039123s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
14372025-01-10T21:56:03.832ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 890.746166ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
14382025-01-10T21:56:03.963ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 909.806342ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
14392025-01-10T21:56:03.983ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 1.020817062s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
14402025-01-10T21:56:04.130ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 955.126519ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
14412025-01-10T21:56:04.226ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 1.148013084s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
14422025-01-10T21:56:04.343ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService28'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte5'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14432025-01-10T21:56:04.343ZINFOSledAgent (ServiceManager): Booting oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
14442025-01-10T21:56:04.392ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 1.234293492s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
14452025-01-10T21:56:04.501ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 809.532615ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
14462025-01-10T21:56:04.513ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 737.689837ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
14472025-01-10T21:56:04.525ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 539.162074ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
14482025-01-10T21:56:04.571ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 62.462312ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
14492025-01-10T21:56:04.646ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 145.47002ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
14502025-01-10T21:56:04.835ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 1.490607459s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
14512025-01-10T21:56:04.846ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 1.380342098s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
14522025-01-10T21:56:04.859ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 186.698132ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
14532025-01-10T21:56:04.871ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 916.114155ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
14542025-01-10T21:56:04.887ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 672.839123ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
14552025-01-10T21:56:04.952ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 1.113132965s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
14562025-01-10T21:56:05.017ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 953.950753ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
14572025-01-10T21:56:05.059ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 231.977246ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
14582025-01-10T21:56:05.077ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 957.956815ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
14592025-01-10T21:56:05.097ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 1.4192524s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
14602025-01-10T21:56:05.260ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 621.99726ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
14612025-01-10T21:56:05.299ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 540.117787ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
14622025-01-10T21:56:05.319ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 621.591037ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
14632025-01-10T21:56:05.384ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 1.292343326s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
14642025-01-10T21:56:05.571ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 583.555957ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
14652025-01-10T21:56:05.637ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 1.149577092s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
14662025-01-10T21:56:05.798ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 740.066535ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
14672025-01-10T21:56:05.850ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 1.072192975s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
14682025-01-10T21:56:05.890ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 1.22121706s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
14692025-01-10T21:56:05.956ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 836.082961ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
14702025-01-10T21:56:05.984ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 1.222627348s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
14712025-01-10T21:56:06.049ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 840.827517ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
14722025-01-10T21:56:06.077ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 1.060785703s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
14732025-01-10T21:56:06.172ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 1.011231843s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
14742025-01-10T21:56:06.238ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 658.53484ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
14752025-01-10T21:56:06.335ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 584.25545ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
14762025-01-10T21:56:06.526ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 1.408298006s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
14772025-01-10T21:56:06.547ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 813.672593ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
14782025-01-10T21:56:06.687ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 1.311028686s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
14792025-01-10T21:56:06.798ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 1.337716478s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
14802025-01-10T21:56:06.806ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 972.06427ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
14812025-01-10T21:56:06.899ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 825.768037ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
14822025-01-10T21:56:06.907ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 1.05875492s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
14832025-01-10T21:56:06.928ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 1.132247929s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
14842025-01-10T21:56:06.936ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 963.514098ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
14852025-01-10T21:56:07.120ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 1.360017558s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
14862025-01-10T21:56:07.146ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 630.981512ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
14872025-01-10T21:56:07.192ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 1.156220062s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
14882025-01-10T21:56:07.215ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 1.360785153s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
14892025-01-10T21:56:07.371ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 641.544799ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
14902025-01-10T21:56:07.735ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 742.69566ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
14912025-01-10T21:56:07.786ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 1.15699106s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
14922025-01-10T21:56:07.792ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 1.194459509s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
14932025-01-10T21:56:07.909ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 611.817935ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
14942025-01-10T21:56:07.942ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 823.443687ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
14952025-01-10T21:56:07.973ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 1.269121126s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
14962025-01-10T21:56:08.007ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 693.061503ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
14972025-01-10T21:56:08.020ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 1.130449612s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
14982025-01-10T21:56:08.069ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 1.385708632s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
14992025-01-10T21:56:08.143ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 1.332962559s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
15002025-01-10T21:56:08.358ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 1.490509917s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
15012025-01-10T21:56:08.489ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 1.353268982s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
15022025-01-10T21:56:08.498ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 1.185578268s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
15032025-01-10T21:56:08.529ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 1.100827436s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
15042025-01-10T21:56:08.583ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 1.188755114s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
15052025-01-10T21:56:08.708ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 1.455645337s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
15062025-01-10T21:56:08.773ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 1.465736864s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
15072025-01-10T21:56:08.952ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 887.383993ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
15082025-01-10T21:56:08.995ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 776.490192ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
15092025-01-10T21:56:09.161ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 962.158376ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
15102025-01-10T21:56:09.252ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 885.17144ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
15112025-01-10T21:56:09.464ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 910.335765ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
15122025-01-10T21:56:09.485ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 938.602395ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
15132025-01-10T21:56:09.638ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 1.428827944s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
15142025-01-10T21:56:09.692ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 956.425851ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
15152025-01-10T21:56:09.780ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 1.313249086s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
15162025-01-10T21:56:09.787ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 582.000009ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
15172025-01-10T21:56:09.848ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 1.201535724s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
15182025-01-10T21:56:09.856ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 911.766809ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
15192025-01-10T21:56:09.864ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 505.120752ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
15202025-01-10T21:56:10.132ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 996.341424ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
15212025-01-10T21:56:10.147ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 1.453005355s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
15222025-01-10T21:56:10.171ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 558.223123ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
15232025-01-10T21:56:10.246ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 770.338458ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
15242025-01-10T21:56:10.286ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 566.962194ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
15252025-01-10T21:56:10.377ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 801.403746ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
15262025-01-10T21:56:10.384ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 561.703078ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
15272025-01-10T21:56:10.393ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 1.183801628s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
15282025-01-10T21:56:10.432ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 1.127243335s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
15292025-01-10T21:56:10.670ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 513.063976ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
15302025-01-10T21:56:10.687ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService21'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::13'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_598e3879-d23d-4354-9e0c-045503f010a6/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::13'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='7eca211b-ad68-4c77-a9aa-042fd8e2fb95'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
15312025-01-10T21:56:10.688ZINFOSledAgent (ServiceManager): Booting oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
15322025-01-10T21:56:10.721ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService17'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::11'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::11'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
15332025-01-10T21:56:10.721ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
15342025-01-10T21:56:10.762ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 983.42825ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
15352025-01-10T21:56:10.782ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 1.267267984s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
15362025-01-10T21:56:10.959ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a") failed: Property not found. retry in 70.037545ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
15372025-01-10T21:56:10.959ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService27'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::9'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte4'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::9]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.6:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
15382025-01-10T21:56:10.967ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc") failed: Property not found. retry in 59.289855ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
15392025-01-10T21:56:10.976ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 977.082491ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
15402025-01-10T21:56:10.976ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
15412025-01-10T21:56:10.980ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService25'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte2'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
15422025-01-10T21:56:10.981ZINFOSledAgent (ServiceManager): Booting oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2
15432025-01-10T21:56:11.029ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 647.067836ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
15442025-01-10T21:56:11.037ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc") failed: Property not found. retry in 58.588408ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
15452025-01-10T21:56:11.044ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a") failed: Property not found. retry in 58.312834ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
15462025-01-10T21:56:11.059ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 1.215956765s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
15472025-01-10T21:56:11.077ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 1.090630808s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
15482025-01-10T21:56:11.104ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 553.248467ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
15492025-01-10T21:56:11.112ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc") failed: Property not found. retry in 167.983919ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
15502025-01-10T21:56:11.119ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a") failed: Property not found. retry in 150.90346ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
15512025-01-10T21:56:11.139ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 654.135776ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
15522025-01-10T21:56:11.150ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2") failed: Property not found. retry in 31.70166ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2
15532025-01-10T21:56:11.188ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 1.377239072s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
15542025-01-10T21:56:11.195ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2") failed: Property not found. retry in 103.613334ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2
15552025-01-10T21:56:11.204ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 1.314900304s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
15562025-01-10T21:56:11.249ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30") failed: Property not found. retry in 74.677735ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
15572025-01-10T21:56:11.281ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a") failed: Property not found. retry in 275.264796ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
15582025-01-10T21:56:11.289ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc") failed: Property not found. retry in 309.729323ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
15592025-01-10T21:56:11.308ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2") failed: Property not found. retry in 217.438795ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2
15602025-01-10T21:56:11.332ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30") failed: Property not found. retry in 82.494754ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
15612025-01-10T21:56:11.427ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30") failed: Property not found. retry in 256.343811ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
15622025-01-10T21:56:11.535ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2") failed: Property not found. retry in 537.958892ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2
15632025-01-10T21:56:11.566ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a") failed: Property not found. retry in 687.83229ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
15642025-01-10T21:56:11.574ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 1.170046866s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
15652025-01-10T21:56:11.587ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 994.450687ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
15662025-01-10T21:56:11.612ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc") failed: Property not found. retry in 555.377182ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
15672025-01-10T21:56:11.621ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 1.011496743s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
15682025-01-10T21:56:11.667ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 752.905591ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
15692025-01-10T21:56:11.687ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 1.070516694s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
15702025-01-10T21:56:11.695ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30") failed: Property not found. retry in 308.073525ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
15712025-01-10T21:56:11.713ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService24'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::c'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte1'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
15722025-01-10T21:56:11.713ZINFOSledAgent (ServiceManager): Booting oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
15732025-01-10T21:56:11.755ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 604.562532ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
15742025-01-10T21:56:11.806ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 1.200387784s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
15752025-01-10T21:56:11.984ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 615.304629ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
15762025-01-10T21:56:11.995ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052") failed: Property not found. retry in 37.897576ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
15772025-01-10T21:56:12.016ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30") failed: Property not found. retry in 421.426441ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
15782025-01-10T21:56:12.043ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052") failed: Property not found. retry in 130.720829ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
15792025-01-10T21:56:12.061ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 626.661829ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
15802025-01-10T21:56:12.083ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2") failed: Property not found. retry in 840.583319ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2
15812025-01-10T21:56:12.178ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc") failed: Property not found. retry in 935.078979ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
15822025-01-10T21:56:12.185ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 761.740388ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
15832025-01-10T21:56:12.191ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052") failed: Property not found. retry in 275.164957ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
15842025-01-10T21:56:12.264ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a") failed: Property not found. retry in 1.489949313s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
15852025-01-10T21:56:12.284ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 1.167557063s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
15862025-01-10T21:56:12.370ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 1.062289402s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
15872025-01-10T21:56:12.429ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 614.508902ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
15882025-01-10T21:56:12.447ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30") failed: Property not found. retry in 1.164358793s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
15892025-01-10T21:56:12.478ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052") failed: Property not found. retry in 518.635825ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
15902025-01-10T21:56:12.528ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 1.3344771s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
15912025-01-10T21:56:12.573ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 669.600985ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
15922025-01-10T21:56:12.590ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 859.190861ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
15932025-01-10T21:56:12.609ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 1.040534499s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
15942025-01-10T21:56:12.642ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 1.451827355s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
15952025-01-10T21:56:12.696ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 868.244285ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
15962025-01-10T21:56:12.754ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 1.002462263s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
15972025-01-10T21:56:12.768ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 905.159686ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
15982025-01-10T21:56:12.934ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2") failed: Property not found. retry in 898.001678ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2
15992025-01-10T21:56:12.956ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 1.499656105s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
16002025-01-10T21:56:13.007ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052") failed: Property not found. retry in 517.842231ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
16012025-01-10T21:56:13.016ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 974.433091ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
16022025-01-10T21:56:13.053ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 708.071113ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
16032025-01-10T21:56:13.122ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc") failed: Property not found. retry in 1.008177458s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
16042025-01-10T21:56:13.318ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 603.880258ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
16052025-01-10T21:56:13.445ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 1.114003559s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
16062025-01-10T21:56:13.460ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 1.4864047s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
16072025-01-10T21:56:13.469ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 1.090173405s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
16082025-01-10T21:56:13.534ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052") failed: Property not found. retry in 863.842485ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
16092025-01-10T21:56:13.574ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 866.388939ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
16102025-01-10T21:56:13.625ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30") failed: Property not found. retry in 807.663911ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
16112025-01-10T21:56:13.660ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 1.427899426s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
16122025-01-10T21:56:13.682ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 1.22599605s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
16132025-01-10T21:56:13.765ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a") failed: Property not found. retry in 984.815942ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
16142025-01-10T21:56:13.772ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 1.447770277s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
16152025-01-10T21:56:13.781ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 822.315431ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
16162025-01-10T21:56:13.841ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2") failed: Property not found. retry in 713.612284ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2
16172025-01-10T21:56:13.873ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 1.163566179s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
16182025-01-10T21:56:13.933ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 524.982989ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
16192025-01-10T21:56:14.004ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 772.372041ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
16202025-01-10T21:56:14.106ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 650.736951ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
16212025-01-10T21:56:14.140ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc") failed: Property not found. retry in 808.32884ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
16222025-01-10T21:56:14.410ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052") failed: Property not found. retry in 1.05898442s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
16232025-01-10T21:56:14.444ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30") failed: Property not found. retry in 839.569972ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
16242025-01-10T21:56:14.456ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 1.097746754s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
16252025-01-10T21:56:14.466ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 503.641672ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
16262025-01-10T21:56:14.477ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 1.343418029s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
16272025-01-10T21:56:14.567ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2") failed: Property not found. retry in 1.172917136s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2
16282025-01-10T21:56:14.576ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 702.931342ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
16292025-01-10T21:56:14.587ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 1.311072104s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
16302025-01-10T21:56:14.614ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 605.24321ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
16312025-01-10T21:56:14.760ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a") failed: Property not found. retry in 1.019765372s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
16322025-01-10T21:56:14.768ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 1.340366694s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
16332025-01-10T21:56:14.787ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 810.102544ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
16342025-01-10T21:56:14.918ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 1.084706514s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
16352025-01-10T21:56:14.960ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 702.320825ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
16362025-01-10T21:56:14.973ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc") failed: Property not found. retry in 973.525389ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
16372025-01-10T21:56:14.984ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 1.052092951s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
16382025-01-10T21:56:15.050ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 558.69289ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
16392025-01-10T21:56:15.102ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 1.067908996s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
16402025-01-10T21:56:15.235ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 1.008271675s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
16412025-01-10T21:56:15.248ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 1.423122686s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
16422025-01-10T21:56:15.293ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 715.171659ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
16432025-01-10T21:56:15.306ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30") failed: Property not found. retry in 942.598473ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
16442025-01-10T21:56:15.403ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService13'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::e'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::e'/> <propval type="astring" name="listen_port" value='8123'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse-admin-single"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::e]:8888'/> <propval type="astring" name="ch_address" value='[fd00:1122:3344:101::e]:9000'/> <propval type="astring" name="ch_binary" value='/opt/oxide/clickhouse/clickhouse'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
16452025-01-10T21:56:15.403ZINFOSledAgent (ServiceManager): Booting oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee
16462025-01-10T21:56:15.484ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052") failed: Property not found. retry in 846.506291ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
16472025-01-10T21:56:15.567ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 594.546891ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
16482025-01-10T21:56:15.612ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 1.432553755s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
16492025-01-10T21:56:15.624ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 660.298652ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
16502025-01-10T21:56:15.643ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee") failed: Property not found. retry in 34.364267ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee
16512025-01-10T21:56:15.676ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 922.867871ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
16522025-01-10T21:56:15.688ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee") failed: Property not found. retry in 87.970834ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee
16532025-01-10T21:56:15.750ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2") failed: Property not found. retry in 1.07152703s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2
16542025-01-10T21:56:15.787ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee") failed: Property not found. retry in 139.333934ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee
16552025-01-10T21:56:15.796ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a") failed: Property not found. retry in 854.830491ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
16562025-01-10T21:56:15.834ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 799.370383ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
16572025-01-10T21:56:16.064ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 935.039415ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
16582025-01-10T21:56:16.077ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 1.215416074s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
16592025-01-10T21:56:16.089ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee") failed: Property not found. retry in 566.903005ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee
16602025-01-10T21:56:16.100ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 790.741658ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
16612025-01-10T21:56:16.110ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc") failed: Property not found. retry in 702.602801ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
16622025-01-10T21:56:16.119ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 1.077347677s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
16632025-01-10T21:56:16.126ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 511.208525ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
16642025-01-10T21:56:16.171ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 514.398756ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
16652025-01-10T21:56:16.182ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 990.576146ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
16662025-01-10T21:56:16.254ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 662.800449ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
16672025-01-10T21:56:16.263ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30") failed: Property not found. retry in 1.469262954s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
16682025-01-10T21:56:16.292ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 1.189909478s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
16692025-01-10T21:56:16.338ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052") failed: Property not found. retry in 1.220803446s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
16702025-01-10T21:56:16.610ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 982.371716ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
16712025-01-10T21:56:16.644ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 1.124158495s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
16722025-01-10T21:56:16.655ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 565.019562ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
16732025-01-10T21:56:16.666ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a") failed: Property not found. retry in 821.176364ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
16742025-01-10T21:56:16.675ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee") failed: Property not found. retry in 711.014283ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee
16752025-01-10T21:56:16.685ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 661.9061ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
16762025-01-10T21:56:16.696ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 1.4316791s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
16772025-01-10T21:56:16.823ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc") failed: Property not found. retry in 743.389037ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
16782025-01-10T21:56:16.832ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2") failed: Property not found. retry in 726.172233ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2
16792025-01-10T21:56:16.901ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 676.714057ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
16802025-01-10T21:56:16.929ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 571.939742ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
16812025-01-10T21:56:17.009ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 731.811363ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
16822025-01-10T21:56:17.058ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 806.418731ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
16832025-01-10T21:56:17.185ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 1.348377079s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
16842025-01-10T21:56:17.209ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 505.706179ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
16852025-01-10T21:56:17.233ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 1.346234322s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
16862025-01-10T21:56:17.305ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 561.167088ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
16872025-01-10T21:56:17.362ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 1.171065961s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
16882025-01-10T21:56:17.402ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee") failed: Property not found. retry in 1.240744047s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee
16892025-01-10T21:56:17.503ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 1.002335829s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
16902025-01-10T21:56:17.523ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 1.177127723s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
16912025-01-10T21:56:17.539ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a") failed: Property not found. retry in 505.108279ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
16922025-01-10T21:56:17.576ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052") failed: Property not found. retry in 596.540416ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
16932025-01-10T21:56:17.591ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2") failed: Property not found. retry in 1.328569399s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2
16942025-01-10T21:56:17.602ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 1.178705006s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
16952025-01-10T21:56:17.615ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc") failed: Property not found. retry in 515.390662ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
16962025-01-10T21:56:17.627ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 819.179636ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
16972025-01-10T21:56:17.728ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 1.444379842s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
16982025-01-10T21:56:17.747ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30") failed: Property not found. retry in 1.422730888s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
16992025-01-10T21:56:17.759ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 836.154042ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
17002025-01-10T21:56:17.781ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 1.023561921s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
17012025-01-10T21:56:17.876ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 1.033152508s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
17022025-01-10T21:56:17.888ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 1.012368323s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
17032025-01-10T21:56:18.054ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a") failed: Property not found. retry in 594.719777ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
17042025-01-10T21:56:18.138ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 1.437125321s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
17052025-01-10T21:56:18.145ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc") failed: Property not found. retry in 1.103646447s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
17062025-01-10T21:56:18.183ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052") failed: Property not found. retry in 735.098348ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
17072025-01-10T21:56:18.461ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 559.331462ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
17082025-01-10T21:56:18.518ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 921.421392ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
17092025-01-10T21:56:18.550ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 994.803287ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
17102025-01-10T21:56:18.562ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 1.384012576s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
17112025-01-10T21:56:18.591ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 1.13903853s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
17122025-01-10T21:56:18.610ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 1.173556579s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
17132025-01-10T21:56:18.662ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee") failed: Property not found. retry in 1.393801819s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee
17142025-01-10T21:56:18.684ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a") failed: Property not found. retry in 1.285342505s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
17152025-01-10T21:56:18.721ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 509.634687ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
17162025-01-10T21:56:18.800ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 650.302404ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
17172025-01-10T21:56:18.823ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 663.865475ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
17182025-01-10T21:56:18.922ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 958.62536ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
17192025-01-10T21:56:18.941ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052") failed: Property not found. retry in 820.913ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
17202025-01-10T21:56:18.959ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2") failed: Property not found. retry in 1.079817229s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2
17212025-01-10T21:56:18.979ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 697.16102ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
17222025-01-10T21:56:19.043ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 739.773121ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
17232025-01-10T21:56:19.187ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30") failed: Property not found. retry in 546.974916ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
17242025-01-10T21:56:19.205ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 1.2065154s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
17252025-01-10T21:56:19.250ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 1.45413775s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
17262025-01-10T21:56:19.263ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc") failed: Property not found. retry in 898.717016ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
17272025-01-10T21:56:19.458ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 1.43000067s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
17282025-01-10T21:56:19.473ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 1.496172789s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
17292025-01-10T21:56:19.501ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 1.376643193s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
17302025-01-10T21:56:19.560ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 1.355973282s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
17312025-01-10T21:56:19.592ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 539.745292ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
17322025-01-10T21:56:19.690ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 950.498244ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
17332025-01-10T21:56:19.750ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 535.477715ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
17342025-01-10T21:56:19.766ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30") failed: Property not found. retry in 671.97599ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
17352025-01-10T21:56:19.781ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052") failed: Property not found. retry in 1.273884545s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
17362025-01-10T21:56:19.795ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 795.305556ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
17372025-01-10T21:56:19.810ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 831.090129ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
17382025-01-10T21:56:19.895ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 1.241063946s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
17392025-01-10T21:56:19.964ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 825.200831ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
17402025-01-10T21:56:19.984ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a") failed: Property not found. retry in 1.027897057s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
17412025-01-10T21:56:20.056ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2") failed: Property not found. retry in 936.508408ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2
17422025-01-10T21:56:20.073ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee") failed: Property not found. retry in 1.179699303s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee
17432025-01-10T21:56:20.145ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 781.411096ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
17442025-01-10T21:56:20.179ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc") failed: Property not found. retry in 922.352089ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
17452025-01-10T21:56:20.303ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 817.753597ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
17462025-01-10T21:56:20.426ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 1.204798137s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
17472025-01-10T21:56:20.454ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30") failed: Property not found. retry in 610.227505ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
17482025-01-10T21:56:20.610ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 571.508053ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
17492025-01-10T21:56:20.655ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 1.054425276s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
17502025-01-10T21:56:20.665ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 1.221933223s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
17512025-01-10T21:56:20.721ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 1.016670615s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
17522025-01-10T21:56:20.805ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 691.320554ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
17532025-01-10T21:56:20.893ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 557.656738ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
17542025-01-10T21:56:20.908ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 1.307163974s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
17552025-01-10T21:56:20.941ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 923.934387ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
17562025-01-10T21:56:20.954ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 1.407539831s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
17572025-01-10T21:56:20.984ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 1.171573304s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
17582025-01-10T21:56:21.006ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2") failed: Property not found. retry in 1.039335635s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2
17592025-01-10T21:56:21.024ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a") failed: Property not found. retry in 1.421142378s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
17602025-01-10T21:56:21.069ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052") failed: Property not found. retry in 611.293029ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
17612025-01-10T21:56:21.081ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30") failed: Property not found. retry in 634.64858ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
17622025-01-10T21:56:21.114ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc") failed: Property not found. retry in 1.481294857s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
17632025-01-10T21:56:21.132ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 1.06698213s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
17642025-01-10T21:56:21.146ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 630.918164ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
17652025-01-10T21:56:21.193ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 1.361047326s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
17662025-01-10T21:56:21.265ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee") failed: Property not found. retry in 826.453843ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee
17672025-01-10T21:56:21.464ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 660.777193ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
17682025-01-10T21:56:21.509ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 967.706483ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
17692025-01-10T21:56:21.641ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 832.83951ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
17702025-01-10T21:56:21.692ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052") failed: Property not found. retry in 890.119731ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
17712025-01-10T21:56:21.723ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 831.61001ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
17722025-01-10T21:56:21.733ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30") failed: Property not found. retry in 1.39918404s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
17732025-01-10T21:56:21.749ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 844.593986ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
17742025-01-10T21:56:21.791ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6") failed: Property not found. retry in 1.491290417s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
17752025-01-10T21:56:21.877ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 541.117948ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
17762025-01-10T21:56:21.898ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 816.396143ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
17772025-01-10T21:56:22.057ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2") failed: Property not found. retry in 1.477627424s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2
17782025-01-10T21:56:22.106ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee") failed: Property not found. retry in 990.510011ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee
17792025-01-10T21:56:22.136ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2") failed: Property not found. retry in 1.358512222s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
17802025-01-10T21:56:22.168ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f") failed: Property not found. retry in 1.11959997s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
17812025-01-10T21:56:22.210ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935") failed: Property not found. retry in 1.470231973s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
17822025-01-10T21:56:22.227ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78") failed: Property not found. retry in 986.543213ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
17832025-01-10T21:56:22.372ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d") failed: Property not found. retry in 1.176220856s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
17842025-01-10T21:56:22.428ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 579.633429ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
17852025-01-10T21:56:22.455ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a") failed: Property not found. retry in 1.383048654s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
17862025-01-10T21:56:22.485ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612") failed: Property not found. retry in 1.073620646s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
17872025-01-10T21:56:22.493ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047") failed: Property not found. retry in 1.402890379s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
17882025-01-10T21:56:22.566ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b") failed: Property not found. retry in 656.205184ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
17892025-01-10T21:56:22.575ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_bce93894-738c-4596-b983-281da931e977") failed: Property not found. retry in 1.404881765s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
17902025-01-10T21:56:22.592ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052") failed: Property not found. retry in 970.472534ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
17912025-01-10T21:56:22.603ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090") failed: Property not found. retry in 549.594608ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
17922025-01-10T21:56:22.611ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc") failed: Property not found. retry in 1.39249804s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
17932025-01-10T21:56:22.724ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f") failed: Property not found. retry in 731.61606ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
17942025-01-10T21:56:23.020ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd") failed: Property not found. retry in 556.391486ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
17952025-01-10T21:56:23.107ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee") failed: Property not found. retry in 1.262743266s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee
17962025-01-10T21:56:24.390ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee") failed: Property not found. retry in 534.945755ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee
17972025-01-10T21:56:24.743ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:38248
17982025-01-10T21:56:24.757ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1606 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:38248 req_id = e1da920b-682c-46b6-b54a-553c8c9a5c32 response_code = 200 uri = /baseboard
17992025-01-10T21:56:24.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee") failed: Property not found. retry in 596.020174ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee
18002025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_f63cd4e6-5f2a-4f8e-bcb3-3013f2844abd
18012025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_nexus_b507bc15-7e7b-4d56-a806-1da438b76052
18022025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_f6578145-7a5a-4961-a28a-6e01465004d6
18032025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_external_dns_bce93894-738c-4596-b983-281da931e977
18042025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_pantry_242b32ff-c26f-43e2-9af7-3c5313c43047
18052025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_clickhouse_e1609ebd-d07d-4215-85da-1ba67a8787ee
18062025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_6e9711ba-8ebd-4b53-ad66-96b22d084d7b
18072025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_nexus_4106fd29-fc2d-4538-b321-136cbc8ea9d2
18082025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_external_dns_f44690fa-03a0-42b0-bb97-26c3d20aac30
18092025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_pantry_b3edd3b4-0e41-4d8f-94d9-5f7fd9b5133f
18102025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_4b7768ac-7d53-4548-864f-2d97118bfb78
18112025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_pantry_d7055447-403c-4747-bc61-71c3d82805fc
18122025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_8774e52d-c771-41cc-a8ab-79387985e935
18132025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_ef867d0b-1af9-4769-a1f4-6030cbcf7090
18142025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_3c40e645-d883-49f8-9f90-44e515e5096d
18152025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_nexus_19558293-ba4c-462e-8746-8d928c3a9612
18162025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_955aa9c6-e944-4ccc-9586-f727632ff66a
18172025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_oximeter_17551ca7-3205-43ba-9a16-734e9902cf2f
18182025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3336 zone = oxz_crucible_fba7a706-948d-4c47-8257-491e17a3a4f2
18192025-01-10T21:56:25.573ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
18202025-01-10T21:56:25.581ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/10481327-13d1-4548-950f-130d063e2690/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
18212025-01-10T21:56:25.583ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 30890226 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:61302 req_id = 2915149b-d0c1-48b6-839e-c16c3be6169e response_code = 204 uri = /omicron-zones
18222025-01-10T21:56:25.587ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1415
18232025-01-10T21:56:25.862ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:790
18242025-01-10T21:56:25.893ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::b]:12221 file = sled-agent/src/rack_setup/service.rs:803
18252025-01-10T21:56:25.952ZINFOSledAgent (RSS): rack_network_config: RackNetworkConfigV2 { bfd: [], bgp: [], infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], autoneg: false, bgp_peers: [], lldp: None, port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, rib_priority: None, vlan_id: None, }, ], switch: Switch0, tx_eq: None, uplink_port_fec: Some( None, ), uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, } file = sled-agent/src/rack_setup/service.rs:1017
18262025-01-10T21:56:25.959ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 2.775567216s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
18272025-01-10T21:56:28.347ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/9e742e19-15eb-4edc-b3b4-60e767ee3ba8/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1079
18282025-01-10T21:56:28.481ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/9e742e19-15eb-4edc-b3b4-60e767ee3ba8/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1079
18292025-01-10T21:56:28.606ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/9e742e19-15eb-4edc-b3b4-60e767ee3ba8/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1079
18302025-01-10T21:56:28.682ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/9e742e19-15eb-4edc-b3b4-60e767ee3ba8/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1079
18312025-01-10T21:56:28.766ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 3.475654243s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
18322025-01-10T21:56:28.846ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/9e742e19-15eb-4edc-b3b4-60e767ee3ba8/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1079
18332025-01-10T21:56:29.136ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/9e742e19-15eb-4edc-b3b4-60e767ee3ba8/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1079
18342025-01-10T21:56:29.321ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/9e742e19-15eb-4edc-b3b4-60e767ee3ba8/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1079
18352025-01-10T21:56:29.604ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/9e742e19-15eb-4edc-b3b4-60e767ee3ba8/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1079
18362025-01-10T21:56:29.924ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/9e742e19-15eb-4edc-b3b4-60e767ee3ba8/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1079
18372025-01-10T21:56:30.300ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/9e742e19-15eb-4edc-b3b4-60e767ee3ba8/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1079
18382025-01-10T21:56:30.580ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/9e742e19-15eb-4edc-b3b4-60e767ee3ba8/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1079
18392025-01-10T21:56:30.906ZINFOSledAgent (RSS): Failed to handoff to nexus: Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/racks/9e742e19-15eb-4edc-b3b4-60e767ee3ba8/initialization-complete) file = sled-agent/src/rack_setup/service.rs:1079
18402025-01-10T21:56:33.397ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "43a68523-00b9-4c71-9115-398eea94ab86", "content-length": "173", "date": "Fri, 10 Jan 2025 21:56:33 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"9e742e19-15eb-4edc-b3b4-60e767ee3ba8\"", request_id: "43a68523-00b9-4c71-9115-398eea94ab86" } file = sled-agent/src/rack_setup/service.rs:1079
18412025-01-10T21:56:34.033ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "40348de3-0e00-40c9-858c-1c60ba1e8f56", "content-length": "173", "date": "Fri, 10 Jan 2025 21:56:33 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"9e742e19-15eb-4edc-b3b4-60e767ee3ba8\"", request_id: "40348de3-0e00-40c9-858c-1c60ba1e8f56" } file = sled-agent/src/rack_setup/service.rs:1079
18422025-01-10T21:56:35.096ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "9efe55ae-b81c-4821-8964-f633a806e025", "content-length": "173", "date": "Fri, 10 Jan 2025 21:56:35 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"9e742e19-15eb-4edc-b3b4-60e767ee3ba8\"", request_id: "9efe55ae-b81c-4821-8964-f633a806e025" } file = sled-agent/src/rack_setup/service.rs:1079
18432025-01-10T21:56:37.294ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "1ea7a064-7d14-4ebd-b4ff-cbb993824999", "content-length": "173", "date": "Fri, 10 Jan 2025 21:56:37 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"2235d715-657e-4a92-8f59-9a6c0c957b73\"", request_id: "1ea7a064-7d14-4ebd-b4ff-cbb993824999" } file = sled-agent/src/rack_setup/service.rs:1079
18442025-01-10T21:56:40.180ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "f31c24ee-f769-42ed-b34b-e3cc77e85708", "content-length": "173", "date": "Fri, 10 Jan 2025 21:56:39 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"2235d715-657e-4a92-8f59-9a6c0c957b73\"", request_id: "f31c24ee-f769-42ed-b34b-e3cc77e85708" } file = sled-agent/src/rack_setup/service.rs:1079
18452025-01-10T21:56:42.900ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "5dde710c-8d05-4cfc-b428-a10e6a683801", "content-length": "173", "date": "Fri, 10 Jan 2025 21:56:42 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"2235d715-657e-4a92-8f59-9a6c0c957b73\"", request_id: "5dde710c-8d05-4cfc-b428-a10e6a683801" } file = sled-agent/src/rack_setup/service.rs:1079
18462025-01-10T21:56:45.063ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "59238bbf-62f1-4b96-a2bd-90eb2c730d2f", "content-length": "173", "date": "Fri, 10 Jan 2025 21:56:44 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"2235d715-657e-4a92-8f59-9a6c0c957b73\"", request_id: "59238bbf-62f1-4b96-a2bd-90eb2c730d2f" } file = sled-agent/src/rack_setup/service.rs:1079
18472025-01-10T21:56:48.547ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "4e47d096-a395-40e0-97a3-36cff5481fd8", "content-length": "173", "date": "Fri, 10 Jan 2025 21:56:48 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"2235d715-657e-4a92-8f59-9a6c0c957b73\"", request_id: "4e47d096-a395-40e0-97a3-36cff5481fd8" } file = sled-agent/src/rack_setup/service.rs:1079
18482025-01-10T21:56:53.689ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "b43ca157-a50d-4fca-b39d-b7228a12924d", "content-length": "173", "date": "Fri, 10 Jan 2025 21:56:53 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"2235d715-657e-4a92-8f59-9a6c0c957b73\"", request_id: "b43ca157-a50d-4fca-b39d-b7228a12924d" } file = sled-agent/src/rack_setup/service.rs:1079
18492025-01-10T21:56:54.796ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:39058
18502025-01-10T21:56:54.796ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 30 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:39058 req_id = bca5b312-7c10-4183-b95a-cccd44ded321 response_code = 200 uri = /baseboard
18512025-01-10T21:56:57.395ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "a68c018c-0621-4e2e-9488-91884066b494", "content-length": "173", "date": "Fri, 10 Jan 2025 21:56:57 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"2235d715-657e-4a92-8f59-9a6c0c957b73\"", request_id: "a68c018c-0621-4e2e-9488-91884066b494" } file = sled-agent/src/rack_setup/service.rs:1079
18522025-01-10T21:57:01.651ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "a48f202d-acac-4d1f-9d35-7acf4d3a06bc", "content-length": "173", "date": "Fri, 10 Jan 2025 21:57:01 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"2235d715-657e-4a92-8f59-9a6c0c957b73\"", request_id: "a48f202d-acac-4d1f-9d35-7acf4d3a06bc" } file = sled-agent/src/rack_setup/service.rs:1079
18532025-01-10T21:57:10.205ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "2ef2e61c-a821-4387-82e1-4d59fc811b85", "content-length": "173", "date": "Fri, 10 Jan 2025 21:57:09 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"2235d715-657e-4a92-8f59-9a6c0c957b73\"", request_id: "2ef2e61c-a821-4387-82e1-4d59fc811b85" } file = sled-agent/src/rack_setup/service.rs:1079
18542025-01-10T21:57:17.684ZINFOSledAgent (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:37589 remote_addr = [fd00:1122:3344:101::d]:40969 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
18552025-01-10T21:57:17.708ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 22080 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = 799f4d46-79ac-4a05-8665-0731d98977bf response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
18562025-01-10T21:57:18.706ZINFOSledAgent (NexusNotifierTask): Successfully put SledAgentInfo to nexus file = sled-agent/src/nexus.rs:500 new_generation = 1 old_generation = 1 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
18572025-01-10T21:57:18.796ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37229
18582025-01-10T21:57:18.797ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
18592025-01-10T21:57:18.797ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
18602025-01-10T21:57:18.797ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-zones.json file = common/src/ledger.rs:177
18612025-01-10T21:57:18.798ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-zones.json file = common/src/ledger.rs:177
18622025-01-10T21:57:18.810ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18632025-01-10T21:57:18.810ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_73828a50-9586-442b-b526-668d3e1de808
18642025-01-10T21:57:18.834ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18652025-01-10T21:57:18.834ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_598e3879-d23d-4354-9e0c-045503f010a6
18662025-01-10T21:57:18.857ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18672025-01-10T21:57:18.857ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_5d7ad0a3-e47e-4369-b599-d02dbd3a8829
18682025-01-10T21:57:18.878ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18692025-01-10T21:57:18.878ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_d27ec511-4e48-4e26-9677-295de0452e91
18702025-01-10T21:57:18.897ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18712025-01-10T21:57:18.897ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_ce5c629d-252d-4e12-9f9d-2087cb772d81
18722025-01-10T21:57:18.917ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18732025-01-10T21:57:18.917ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_7b0e1b76-af4f-49bd-9848-611d8d8666ad
18742025-01-10T21:57:18.937ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18752025-01-10T21:57:18.937ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_619e5037-b196-43f6-b092-666b1b1387ff
18762025-01-10T21:57:18.957ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18772025-01-10T21:57:18.957ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_eecab578-d379-46e4-90f5-9dc11839dad5
18782025-01-10T21:57:18.976ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18792025-01-10T21:57:18.976ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_784c4a6b-3804-44fa-afd4-c974bef9dd5d
18802025-01-10T21:57:18.987ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 191474 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37229 req_id = 28219e35-cdf1-425e-bf1d-4d9987e1c345 response_code = 200 uri = /inventory
18812025-01-10T21:57:20.961ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61516
18822025-01-10T21:57:20.964ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6decc476-dbff-4cb1-a909-d9d4408adaab, kind: Service { id: f44690fa-03a0-42b0-bb97-26c3d20aac30 }, name: Name("external-dns-f44690fa-03a0-42b0-bb97-26c3d20aac30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 143])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6f22ade2-da2d-4cbc-9141-adcc3b335aed, kind: Service { id: bce93894-738c-4596-b983-281da931e977 }, name: Name("external-dns-bce93894-738c-4596-b983-281da931e977"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 88])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 8448bae1-35b5-443a-8e61-6f2848603054, kind: Service { id: 4106fd29-fc2d-4538-b321-136cbc8ea9d2 }, name: Name("nexus-4106fd29-fc2d-4538-b321-136cbc8ea9d2"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 27])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9676df2d-b2c5-40f4-ae20-30ad4de6c8e2, kind: Service { id: 19558293-ba4c-462e-8746-8d928c3a9612 }, name: Name("nexus-19558293-ba4c-462e-8746-8d928c3a9612"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 42])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bf22f522-6412-4896-90b5-9bc8aab0192d, kind: Service { id: b507bc15-7e7b-4d56-a806-1da438b76052 }, name: Name("nexus-b507bc15-7e7b-4d56-a806-1da438b76052"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 137])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
18832025-01-10T21:57:20.964ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18842025-01-10T21:57:20.965ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18852025-01-10T21:57:20.965ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18862025-01-10T21:57:20.965ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18872025-01-10T21:57:20.965ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
18882025-01-10T21:57:20.965ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18892025-01-10T21:57:20.965ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3415 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61516 req_id = a7649f2d-69be-4a6b-8a48-7898b870cf72 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18902025-01-10T21:57:20.967ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:1089
18912025-01-10T21:57:20.971ZINFOSledAgent (RSS): Writing ledger to /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
18922025-01-10T21:57:20.971ZINFOSledAgent (RSS): Writing ledger to /pool/int/10481327-13d1-4548-950f-130d063e2690/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
18932025-01-10T21:57:21.186ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54522
18942025-01-10T21:57:21.186ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
18952025-01-10T21:57:21.186ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
18962025-01-10T21:57:21.186ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-zones.json file = common/src/ledger.rs:177
18972025-01-10T21:57:21.187ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-zones.json file = common/src/ledger.rs:177
18982025-01-10T21:57:21.199ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18992025-01-10T21:57:21.199ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_73828a50-9586-442b-b526-668d3e1de808
19002025-01-10T21:57:21.224ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
19012025-01-10T21:57:21.224ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_598e3879-d23d-4354-9e0c-045503f010a6
19022025-01-10T21:57:21.250ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
19032025-01-10T21:57:21.250ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_5d7ad0a3-e47e-4369-b599-d02dbd3a8829
19042025-01-10T21:57:21.273ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
19052025-01-10T21:57:21.273ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_d27ec511-4e48-4e26-9677-295de0452e91
19062025-01-10T21:57:21.296ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
19072025-01-10T21:57:21.296ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_ce5c629d-252d-4e12-9f9d-2087cb772d81
19082025-01-10T21:57:21.317ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
19092025-01-10T21:57:21.317ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_7b0e1b76-af4f-49bd-9848-611d8d8666ad
19102025-01-10T21:57:21.337ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
19112025-01-10T21:57:21.337ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_619e5037-b196-43f6-b092-666b1b1387ff
19122025-01-10T21:57:21.353ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55752
19132025-01-10T21:57:21.353ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6decc476-dbff-4cb1-a909-d9d4408adaab, kind: Service { id: f44690fa-03a0-42b0-bb97-26c3d20aac30 }, name: Name("external-dns-f44690fa-03a0-42b0-bb97-26c3d20aac30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 143])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6f22ade2-da2d-4cbc-9141-adcc3b335aed, kind: Service { id: bce93894-738c-4596-b983-281da931e977 }, name: Name("external-dns-bce93894-738c-4596-b983-281da931e977"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 88])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 8448bae1-35b5-443a-8e61-6f2848603054, kind: Service { id: 4106fd29-fc2d-4538-b321-136cbc8ea9d2 }, name: Name("nexus-4106fd29-fc2d-4538-b321-136cbc8ea9d2"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 27])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9676df2d-b2c5-40f4-ae20-30ad4de6c8e2, kind: Service { id: 19558293-ba4c-462e-8746-8d928c3a9612 }, name: Name("nexus-19558293-ba4c-462e-8746-8d928c3a9612"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 42])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bf22f522-6412-4896-90b5-9bc8aab0192d, kind: Service { id: b507bc15-7e7b-4d56-a806-1da438b76052 }, name: Name("nexus-b507bc15-7e7b-4d56-a806-1da438b76052"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 137])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
19142025-01-10T21:57:21.353ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19152025-01-10T21:57:21.353ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19162025-01-10T21:57:21.353ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19172025-01-10T21:57:21.353ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19182025-01-10T21:57:21.353ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
19192025-01-10T21:57:21.354ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19202025-01-10T21:57:21.354ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 394 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55752 req_id = e80ff8df-179b-41c1-a077-42b1335daece response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19212025-01-10T21:57:21.358ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
19222025-01-10T21:57:21.358ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_eecab578-d379-46e4-90f5-9dc11839dad5
19232025-01-10T21:57:21.377ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
19242025-01-10T21:57:21.377ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_784c4a6b-3804-44fa-afd4-c974bef9dd5d
19252025-01-10T21:57:21.389ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 202550 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54522 req_id = 7a969fd0-fe15-46ba-8080-ffd5b2c59d23 response_code = 200 uri = /inventory
19262025-01-10T21:57:22.096ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41101
19272025-01-10T21:57:22.096ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6decc476-dbff-4cb1-a909-d9d4408adaab, kind: Service { id: f44690fa-03a0-42b0-bb97-26c3d20aac30 }, name: Name("external-dns-f44690fa-03a0-42b0-bb97-26c3d20aac30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 143])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6f22ade2-da2d-4cbc-9141-adcc3b335aed, kind: Service { id: bce93894-738c-4596-b983-281da931e977 }, name: Name("external-dns-bce93894-738c-4596-b983-281da931e977"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 88])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 8448bae1-35b5-443a-8e61-6f2848603054, kind: Service { id: 4106fd29-fc2d-4538-b321-136cbc8ea9d2 }, name: Name("nexus-4106fd29-fc2d-4538-b321-136cbc8ea9d2"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 27])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9676df2d-b2c5-40f4-ae20-30ad4de6c8e2, kind: Service { id: 19558293-ba4c-462e-8746-8d928c3a9612 }, name: Name("nexus-19558293-ba4c-462e-8746-8d928c3a9612"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 42])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bf22f522-6412-4896-90b5-9bc8aab0192d, kind: Service { id: b507bc15-7e7b-4d56-a806-1da438b76052 }, name: Name("nexus-b507bc15-7e7b-4d56-a806-1da438b76052"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 137])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
19282025-01-10T21:57:22.096ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19292025-01-10T21:57:22.096ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19302025-01-10T21:57:22.096ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19312025-01-10T21:57:22.096ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19322025-01-10T21:57:22.096ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
19332025-01-10T21:57:22.096ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19342025-01-10T21:57:22.096ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 300 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41101 req_id = cea836be-8cfa-46c8-a3a0-bf706cc119a4 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19352025-01-10T21:57:22.821ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59238
19362025-01-10T21:57:22.821ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6decc476-dbff-4cb1-a909-d9d4408adaab, kind: Service { id: f44690fa-03a0-42b0-bb97-26c3d20aac30 }, name: Name("external-dns-f44690fa-03a0-42b0-bb97-26c3d20aac30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 143])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6f22ade2-da2d-4cbc-9141-adcc3b335aed, kind: Service { id: bce93894-738c-4596-b983-281da931e977 }, name: Name("external-dns-bce93894-738c-4596-b983-281da931e977"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 88])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 8448bae1-35b5-443a-8e61-6f2848603054, kind: Service { id: 4106fd29-fc2d-4538-b321-136cbc8ea9d2 }, name: Name("nexus-4106fd29-fc2d-4538-b321-136cbc8ea9d2"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 27])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9676df2d-b2c5-40f4-ae20-30ad4de6c8e2, kind: Service { id: 19558293-ba4c-462e-8746-8d928c3a9612 }, name: Name("nexus-19558293-ba4c-462e-8746-8d928c3a9612"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 42])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bf22f522-6412-4896-90b5-9bc8aab0192d, kind: Service { id: b507bc15-7e7b-4d56-a806-1da438b76052 }, name: Name("nexus-b507bc15-7e7b-4d56-a806-1da438b76052"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 137])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
19372025-01-10T21:57:22.821ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19382025-01-10T21:57:22.821ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19392025-01-10T21:57:22.822ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19402025-01-10T21:57:22.822ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19412025-01-10T21:57:22.822ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
19422025-01-10T21:57:22.822ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19432025-01-10T21:57:22.822ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 316 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59238 req_id = 7814dae8-e2d0-4fcc-be9e-0faaadfb7574 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19442025-01-10T21:57:24.830ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:49897
19452025-01-10T21:57:24.831ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 30 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:49897 req_id = fdd54a94-41db-4b6f-8442-5464af8abc3f response_code = 200 uri = /baseboard
19462025-01-10T21:57:27.366ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47062
19472025-01-10T21:57:27.366ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 304 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47062 req_id = 60f0e487-4cf4-4c7a-a4fd-7b216fecf76b response_code = 200 uri = /vpc-routes
19482025-01-10T21:57:27.399ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
19492025-01-10T21:57:27.399ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 700 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:47062 req_id = 5ac8d824-6d35-4599-9843-f3042b787cb7 response_code = 204 uri = /eip-gateways
19502025-01-10T21:57:27.488ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2415 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:47062 req_id = 318f7c03-0b40-4f2f-b850-301a4c594664 response_code = 204 uri = /vpc-routes
19512025-01-10T21:57:35.264ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53077
19522025-01-10T21:57:35.265ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 50 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53077 req_id = 9f1b0d13-1410-4632-9442-cc7102be04d6 response_code = 200 uri = /vpc-routes
19532025-01-10T21:57:35.426ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
19542025-01-10T21:57:35.426ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53077 req_id = d9880b28-c6b3-48b4-ae87-1f959bae8e12 response_code = 204 uri = /eip-gateways
19552025-01-10T21:57:35.430ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58519
19562025-01-10T21:57:35.431ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 966 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58519 req_id = a81bdd83-a325-4ee2-b896-c6f052e03a18 response_code = 200 uri = /v2p
19572025-01-10T21:57:35.436ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 144 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53077 req_id = 28b7ecb6-4639-4c03-a671-eb8eb90621cf response_code = 204 uri = /vpc-routes
19582025-01-10T21:57:35.501ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58703
19592025-01-10T21:57:35.502ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
19602025-01-10T21:57:35.558ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
19612025-01-10T21:57:35.614ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113203 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:58703 req_id = 9ef3d5b3-e432-4bed-b671-4befd4a9a7d4 response_code = 204 uri = /switch-ports
19622025-01-10T21:57:35.619ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2030 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58703 req_id = e6a45ca6-1673-4ab0-8b16-d1de23bdd718 response_code = 200 uri = /network-bootstore-config
19632025-01-10T21:57:35.657ZINFOSledAgent (bootstore): Attempting to update network config with generation: 2, current_generation: 1 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
19642025-01-10T21:57:35.657ZINFOSledAgent (bootstore): Writing ledger to /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
19652025-01-10T21:57:35.658ZINFOSledAgent (bootstore): Writing ledger to /pool/int/10481327-13d1-4548-950f-130d063e2690/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
19662025-01-10T21:57:35.660ZINFOSledAgent (bootstore): Broadcasting network config with generation 2 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
19672025-01-10T21:57:35.660ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5928 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:58703 req_id = c98b6814-cd7b-4b3f-811d-f52a1ab14753 response_code = 204 uri = /network-bootstore-config
19682025-01-10T21:57:36.046ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47523
19692025-01-10T21:57:36.046ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47523 req_id = e94b2722-369e-4cdf-8580-b0186854934a response_code = 200 uri = /vpc-routes
19702025-01-10T21:57:36.083ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50208
19712025-01-10T21:57:36.083ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 25 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50208 req_id = 92506623-ed2d-4bc5-ab20-460fbf257619 response_code = 200 uri = /vpc-routes
19722025-01-10T21:57:36.202ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
19732025-01-10T21:57:36.202ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47523 req_id = 06d502b8-c605-4aa7-aab2-580440b7e238 response_code = 204 uri = /eip-gateways
19742025-01-10T21:57:36.217ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 216 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47523 req_id = 62a7df83-5830-4ebe-bef8-c294e7eaed16 response_code = 204 uri = /vpc-routes
19752025-01-10T21:57:36.255ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
19762025-01-10T21:57:36.255ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:50208 req_id = 965cf4a7-3a46-4579-910d-ad3d8cb86a57 response_code = 204 uri = /eip-gateways
19772025-01-10T21:57:36.255ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59587
19782025-01-10T21:57:36.255ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
19792025-01-10T21:57:36.294ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
19802025-01-10T21:57:36.349ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93554 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59587 req_id = 5d7fb319-c717-426c-82ea-77ab927d10c5 response_code = 204 uri = /switch-ports
19812025-01-10T21:57:36.349ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51045
19822025-01-10T21:57:36.350ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 246 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59587 req_id = ac9f3705-96e3-4ce5-afcb-fbf2c2ea4a39 response_code = 200 uri = /network-bootstore-config
19832025-01-10T21:57:36.350ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 234 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51045 req_id = 24031d4c-be5f-4b53-a160-fa7932834293 response_code = 200 uri = /v2p
19842025-01-10T21:57:36.350ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 449 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:50208 req_id = 74f78e62-25a6-45ef-921e-ae3ee1ec321d response_code = 204 uri = /vpc-routes
19852025-01-10T21:57:36.451ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:65373
19862025-01-10T21:57:36.451ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 162 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:65373 req_id = 07e6cf6d-b43f-4827-ada5-d8a582ca4c79 response_code = 200 uri = /v2p
19872025-01-10T21:57:36.464ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34539
19882025-01-10T21:57:36.464ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
19892025-01-10T21:57:36.500ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
19902025-01-10T21:57:36.552ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87326 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:34539 req_id = fdbb834d-ef25-402b-9457-fbbd89a5dde2 response_code = 204 uri = /switch-ports
19912025-01-10T21:57:36.552ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34539 req_id = ed0f2edb-9b62-4054-a16a-24c757c1f3fa response_code = 200 uri = /network-bootstore-config
19922025-01-10T21:57:47.691ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7437 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = 6f8da447-61e8-4b6a-8254-6d71d72f5bfa response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
19932025-01-10T21:57:54.864ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:59811
19942025-01-10T21:57:54.865ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 43 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:59811 req_id = c756b0a6-4c67-4470-92ef-a9ecf85e12d1 response_code = 200 uri = /baseboard
19952025-01-10T21:58:05.073ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63636
19962025-01-10T21:58:05.073ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63636 req_id = ef6eaeb4-b7a8-4c5e-a38e-cd9ae85290d3 response_code = 200 uri = /vpc-routes
19972025-01-10T21:58:05.188ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
19982025-01-10T21:58:05.188ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63636 req_id = e2488094-ea84-4d97-b9ed-5188d2e0d0e9 response_code = 204 uri = /eip-gateways
19992025-01-10T21:58:05.217ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58704
20002025-01-10T21:58:05.217ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58704 req_id = e2d906c4-7625-4a44-bc0d-f203c7a1ee31 response_code = 200 uri = /v2p
20012025-01-10T21:58:05.223ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 200 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63636 req_id = c24346db-ef56-4b05-b575-396c8080f2c7 response_code = 204 uri = /vpc-routes
20022025-01-10T21:58:05.395ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40072
20032025-01-10T21:58:05.395ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
20042025-01-10T21:58:05.430ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
20052025-01-10T21:58:05.488ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93618 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:40072 req_id = 805bef7c-66b3-4c13-a260-544ef204a939 response_code = 204 uri = /switch-ports
20062025-01-10T21:58:05.489ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40072 req_id = 49af6f73-3264-48ae-b227-ead359961734 response_code = 200 uri = /network-bootstore-config
20072025-01-10T21:58:06.043ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40280
20082025-01-10T21:58:06.043ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 77 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40280 req_id = b940cd31-0d64-4c46-8dc8-67dc4ac63a4c response_code = 200 uri = /vpc-routes
20092025-01-10T21:58:06.045ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33133
20102025-01-10T21:58:06.045ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33133 req_id = 231951c7-2771-4b97-9d1d-50275558f03f response_code = 200 uri = /v2p
20112025-01-10T21:58:06.058ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
20122025-01-10T21:58:06.058ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40280 req_id = e879076a-7edb-4986-952b-6f026d26d513 response_code = 204 uri = /eip-gateways
20132025-01-10T21:58:06.069ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 190 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40280 req_id = fb416656-6764-4bdc-a766-5e9c1778a8a7 response_code = 204 uri = /vpc-routes
20142025-01-10T21:58:06.114ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37901
20152025-01-10T21:58:06.115ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
20162025-01-10T21:58:06.153ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
20172025-01-10T21:58:06.206ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91453 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:37901 req_id = 85297d57-dcd4-4a8f-aa8d-2d327165d8d3 response_code = 204 uri = /switch-ports
20182025-01-10T21:58:06.206ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43219
20192025-01-10T21:58:06.206ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38735
20202025-01-10T21:58:06.206ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 212 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38735 req_id = c1c2c8a3-930a-4d32-8485-0ec04fb8a15f response_code = 200 uri = /v2p
20212025-01-10T21:58:06.206ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 228 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37901 req_id = c42ca45a-83a3-4fbf-b578-2a521a087a41 response_code = 200 uri = /network-bootstore-config
20222025-01-10T21:58:06.207ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43219 req_id = 89828f43-e30f-43a0-82bd-07adcf974386 response_code = 200 uri = /vpc-routes
20232025-01-10T21:58:06.247ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
20242025-01-10T21:58:06.247ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43219 req_id = 3366ccee-d792-41aa-831a-c55daf1d5a69 response_code = 204 uri = /eip-gateways
20252025-01-10T21:58:06.252ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:65137
20262025-01-10T21:58:06.252ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
20272025-01-10T21:58:06.288ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
20282025-01-10T21:58:06.342ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90411 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:65137 req_id = 8c98b558-34c5-43ab-8802-94146fdc47f3 response_code = 204 uri = /switch-ports
20292025-01-10T21:58:06.343ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 254 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43219 req_id = f9df59bf-1398-4fde-aa81-e481cbc04bc2 response_code = 204 uri = /vpc-routes
20302025-01-10T21:58:06.343ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 81 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:65137 req_id = f1136599-1f41-4904-b3fc-d10f6253f611 response_code = 200 uri = /network-bootstore-config
20312025-01-10T21:58:17.691ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7461 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = 94d69c5a-b69f-48ac-8798-6e6f8f53b01f response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
20322025-01-10T21:58:24.899ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47904
20332025-01-10T21:58:24.900ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 34 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47904 req_id = 62aa7e6c-1272-4139-ad38-05a7fcd51e0d response_code = 200 uri = /baseboard
20342025-01-10T21:58:35.184ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44810
20352025-01-10T21:58:35.185ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44810 req_id = 51545bf1-656e-4ce3-a6a1-35014c9779b3 response_code = 200 uri = /vpc-routes
20362025-01-10T21:58:35.242ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
20372025-01-10T21:58:35.242ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44810 req_id = 7ffa0ad5-e2cd-48ac-a1da-b6f0436a1fa0 response_code = 204 uri = /eip-gateways
20382025-01-10T21:58:35.361ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 182 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44810 req_id = c55fc2b1-be53-4fef-a26c-05163ac28c48 response_code = 204 uri = /vpc-routes
20392025-01-10T21:58:35.393ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44640
20402025-01-10T21:58:35.393ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
20412025-01-10T21:58:35.432ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
20422025-01-10T21:58:35.487ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94508 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:44640 req_id = 6d0713ef-8c7d-4c63-8713-00fc23fdca7f response_code = 204 uri = /switch-ports
20432025-01-10T21:58:35.488ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60301
20442025-01-10T21:58:35.488ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60301 req_id = b4d3708c-512d-407c-8dd4-35d31a8deb18 response_code = 200 uri = /v2p
20452025-01-10T21:58:35.488ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44640 req_id = 5dd6925d-baf4-4ee5-a7f0-3e95498b3b44 response_code = 200 uri = /network-bootstore-config
20462025-01-10T21:58:35.975ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63634
20472025-01-10T21:58:35.975ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63634 req_id = 66a10a82-1eb6-4316-9d46-b141c62454ca response_code = 200 uri = /vpc-routes
20482025-01-10T21:58:36.023ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
20492025-01-10T21:58:36.023ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63634 req_id = bad4cbdc-02c6-4d5c-9f2d-4abb7a834aca response_code = 204 uri = /eip-gateways
20502025-01-10T21:58:36.036ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 256 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63634 req_id = f7afb7cb-d50f-4559-9ca5-f03115c5ef80 response_code = 204 uri = /vpc-routes
20512025-01-10T21:58:36.068ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42835
20522025-01-10T21:58:36.069ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42835 req_id = e30f9079-155a-4717-9819-f47b1fb9a181 response_code = 200 uri = /v2p
20532025-01-10T21:58:36.381ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48930
20542025-01-10T21:58:36.381ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48930 req_id = 543afc4f-7002-40ca-85d8-09de717c115b response_code = 200 uri = /vpc-routes
20552025-01-10T21:58:36.426ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
20562025-01-10T21:58:36.426ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48930 req_id = 8306ffad-c7ac-468d-90c2-5119b40cd633 response_code = 204 uri = /eip-gateways
20572025-01-10T21:58:36.431ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59250
20582025-01-10T21:58:36.431ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
20592025-01-10T21:58:36.469ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
20602025-01-10T21:58:36.521ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90631 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59250 req_id = bb28769f-f2c2-46fd-9cad-86f990f113da response_code = 204 uri = /switch-ports
20612025-01-10T21:58:36.522ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52898
20622025-01-10T21:58:36.522ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 199 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52898 req_id = a1595634-e115-4e10-89ff-423113a8b92c response_code = 200 uri = /v2p
20632025-01-10T21:58:36.522ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 325 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48930 req_id = 97bc459d-cac9-4591-9e61-493c5d6ebeae response_code = 204 uri = /vpc-routes
20642025-01-10T21:58:36.522ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 239 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59250 req_id = b15069a4-f9d6-47e6-b9e5-0cc860053ef5 response_code = 200 uri = /network-bootstore-config
20652025-01-10T21:58:36.586ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60908
20662025-01-10T21:58:36.586ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
20672025-01-10T21:58:36.623ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
20682025-01-10T21:58:36.679ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92617 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:60908 req_id = 14daaba9-b1bb-4a57-afad-f798cef69b9d response_code = 204 uri = /switch-ports
20692025-01-10T21:58:36.679ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 178 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60908 req_id = 0516a026-ffa9-4cc0-bbf0-2946074c84aa response_code = 200 uri = /network-bootstore-config
20702025-01-10T21:58:47.693ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7762 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = a5bd2e40-ff11-414e-84fa-6640f1c440e6 response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
20712025-01-10T21:58:54.934ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:61774
20722025-01-10T21:58:54.934ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:61774 req_id = eff9feb4-32c7-401c-954a-45185ff89b49 response_code = 200 uri = /baseboard
20732025-01-10T21:59:05.031ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56864
20742025-01-10T21:59:05.031ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 72 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56864 req_id = 0bb204c5-636a-415c-930b-781b322346a2 response_code = 200 uri = /vpc-routes
20752025-01-10T21:59:05.078ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
20762025-01-10T21:59:05.078ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 77 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56864 req_id = 5fb139b9-77d4-4d2b-8c2d-c8c18d641fc9 response_code = 204 uri = /eip-gateways
20772025-01-10T21:59:05.116ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58240
20782025-01-10T21:59:05.116ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
20792025-01-10T21:59:05.153ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
20802025-01-10T21:59:05.207ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90494 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:58240 req_id = 0396fa71-70a1-4ea7-81ee-2dcf36ce9887 response_code = 204 uri = /switch-ports
20812025-01-10T21:59:05.207ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49213
20822025-01-10T21:59:05.207ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 236 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49213 req_id = ad9f8111-cc2e-46e3-948f-cd64dbbb9798 response_code = 200 uri = /v2p
20832025-01-10T21:59:05.207ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 252 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58240 req_id = d0fa639d-76e5-4944-8d26-e76ed25d5f3f response_code = 200 uri = /network-bootstore-config
20842025-01-10T21:59:05.208ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 494 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56864 req_id = 9eb17990-554c-4672-be90-3be17d2aafea response_code = 204 uri = /vpc-routes
20852025-01-10T21:59:05.880ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50337
20862025-01-10T21:59:05.880ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50337 req_id = 9dfb0cd3-84ac-41b2-89b1-bfacfaf56566 response_code = 200 uri = /vpc-routes
20872025-01-10T21:59:05.898ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53732
20882025-01-10T21:59:05.899ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53732 req_id = ae411686-cab5-449d-a8ec-7a799e207a27 response_code = 200 uri = /v2p
20892025-01-10T21:59:05.914ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
20902025-01-10T21:59:05.915ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50337 req_id = 3d17f7cc-de65-49da-af7e-e4b06ef4d32f response_code = 204 uri = /eip-gateways
20912025-01-10T21:59:06.035ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 365 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50337 req_id = 9ab934b0-a8ac-40b9-894a-ac81dc325b55 response_code = 204 uri = /vpc-routes
20922025-01-10T21:59:06.105ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41305
20932025-01-10T21:59:06.105ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41305 req_id = 8867fd9b-2937-485e-917e-4a30e4aa5dfb response_code = 200 uri = /v2p
20942025-01-10T21:59:06.159ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54503
20952025-01-10T21:59:06.159ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
20962025-01-10T21:59:06.198ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
20972025-01-10T21:59:06.254ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95002 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:54503 req_id = 3ee40b48-b6b5-4db5-8fb9-da017d603e4b response_code = 204 uri = /switch-ports
20982025-01-10T21:59:06.254ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54778
20992025-01-10T21:59:06.254ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 50 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54778 req_id = 7a75276a-87f3-41be-8295-84a0bc298408 response_code = 200 uri = /vpc-routes
21002025-01-10T21:59:06.254ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54503 req_id = b8538c70-d7f0-4b72-8820-f0c6aeda1697 response_code = 200 uri = /network-bootstore-config
21012025-01-10T21:59:06.302ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
21022025-01-10T21:59:06.302ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:54778 req_id = 353d193f-e02d-474f-aaa2-94bb3c265f64 response_code = 204 uri = /eip-gateways
21032025-01-10T21:59:06.325ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62863
21042025-01-10T21:59:06.325ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
21052025-01-10T21:59:06.360ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
21062025-01-10T21:59:06.415ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89807 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:62863 req_id = ab3c0530-a626-4fff-beff-c6c659a07586 response_code = 204 uri = /switch-ports
21072025-01-10T21:59:06.415ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 232 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:54778 req_id = e0525610-34a3-4d61-916e-f4eb8cd3b2de response_code = 204 uri = /vpc-routes
21082025-01-10T21:59:06.415ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62863 req_id = 8e23e59e-5207-47ef-a2ee-aa041e66ee80 response_code = 200 uri = /network-bootstore-config
21092025-01-10T21:59:17.691ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6820 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = e47c62c1-272e-4083-bd44-06a4d39a01bb response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
21102025-01-10T21:59:24.965ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:39337
21112025-01-10T21:59:24.966ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 28 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:39337 req_id = 53e91cb8-a179-4ec5-9648-25de17faab2f response_code = 200 uri = /baseboard
21122025-01-10T21:59:35.196ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41959
21132025-01-10T21:59:35.196ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41959 req_id = a2053daf-3228-4bad-ac04-672cc9ad5b6a response_code = 200 uri = /vpc-routes
21142025-01-10T21:59:35.238ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
21152025-01-10T21:59:35.238ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41959 req_id = 01321835-5d0d-4494-9ef8-38186c805b8d response_code = 204 uri = /eip-gateways
21162025-01-10T21:59:35.247ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 199 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41959 req_id = 1ec78aed-94d1-4496-8bfe-0fbef79060cb response_code = 204 uri = /vpc-routes
21172025-01-10T21:59:35.324ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54533
21182025-01-10T21:59:35.324ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 140 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54533 req_id = 17f6ca7c-2532-4a92-ad1f-519edb884f54 response_code = 200 uri = /v2p
21192025-01-10T21:59:35.482ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57201
21202025-01-10T21:59:35.482ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
21212025-01-10T21:59:35.518ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
21222025-01-10T21:59:35.571ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88584 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:57201 req_id = 1d743d01-1491-442a-9821-7232b878092a response_code = 204 uri = /switch-ports
21232025-01-10T21:59:35.571ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57201 req_id = 51e1738a-6dd0-4678-94b4-03e1757bcdd8 response_code = 200 uri = /network-bootstore-config
21242025-01-10T21:59:36.103ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48377
21252025-01-10T21:59:36.104ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48377 req_id = ecbab9f3-cccb-4b53-8f79-a18e3c1f805d response_code = 200 uri = /vpc-routes
21262025-01-10T21:59:36.136ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63394
21272025-01-10T21:59:36.136ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63394 req_id = b2d10383-12de-4ced-8f14-175d8a7fd518 response_code = 200 uri = /v2p
21282025-01-10T21:59:36.144ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
21292025-01-10T21:59:36.144ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48377 req_id = 7ec5bbc9-a7bd-43be-883d-fe573295cb05 response_code = 204 uri = /eip-gateways
21302025-01-10T21:59:36.154ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 161 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48377 req_id = 52aca992-115b-40dc-9c48-d7f1dc16c058 response_code = 204 uri = /vpc-routes
21312025-01-10T21:59:36.273ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38315
21322025-01-10T21:59:36.273ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
21332025-01-10T21:59:36.311ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
21342025-01-10T21:59:36.365ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92271 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:38315 req_id = 642056cf-3d60-44d3-a1b5-a2a0a4653681 response_code = 204 uri = /switch-ports
21352025-01-10T21:59:36.366ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50418
21362025-01-10T21:59:36.366ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 165 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38315 req_id = adee0230-b0d4-41bd-ac05-5c0d08ec23b2 response_code = 200 uri = /network-bootstore-config
21372025-01-10T21:59:36.366ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50418 req_id = 0c40d0c5-723f-493f-878b-c358825c94e9 response_code = 200 uri = /vpc-routes
21382025-01-10T21:59:36.385ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
21392025-01-10T21:59:36.385ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:50418 req_id = 977875c1-4c57-4cbc-8bef-4f528a5d013a response_code = 204 uri = /eip-gateways
21402025-01-10T21:59:36.402ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51746
21412025-01-10T21:59:36.403ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51746 req_id = 4262f10c-98f2-4ea5-8b07-8269117ba471 response_code = 200 uri = /v2p
21422025-01-10T21:59:36.411ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 263 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:50418 req_id = 52394c0d-4339-4db5-8145-533230eb98ee response_code = 204 uri = /vpc-routes
21432025-01-10T21:59:36.489ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33075
21442025-01-10T21:59:36.490ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
21452025-01-10T21:59:36.525ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
21462025-01-10T21:59:36.576ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86591 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:33075 req_id = a01da7be-4b6b-44f5-afa3-adb8dac10e32 response_code = 204 uri = /switch-ports
21472025-01-10T21:59:36.576ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33075 req_id = 1b8666c9-d291-4e40-9e17-25c419483420 response_code = 200 uri = /network-bootstore-config
21482025-01-10T21:59:42.960ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
21492025-01-10T21:59:47.692ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7171 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = 5eedabe1-1a30-4856-aabb-f9cd2bd160f8 response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
21502025-01-10T21:59:54.998ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:38802
21512025-01-10T21:59:54.998ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 31 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:38802 req_id = 46083263-12ad-4cf7-b75d-36d5f06ea336 response_code = 200 uri = /baseboard
21522025-01-10T22:00:05.030ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34515
21532025-01-10T22:00:05.030ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34515 req_id = 6c95fe0f-3a95-43e8-8d80-e0a0264164eb response_code = 200 uri = /vpc-routes
21542025-01-10T22:00:05.073ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
21552025-01-10T22:00:05.073ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 71 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34515 req_id = 97295602-1bb5-45a3-8f20-10c2f3af4b31 response_code = 204 uri = /eip-gateways
21562025-01-10T22:00:05.082ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 164 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34515 req_id = 3677dc86-e64e-4bca-a09f-c47db43fd2c5 response_code = 204 uri = /vpc-routes
21572025-01-10T22:00:05.170ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52402
21582025-01-10T22:00:05.170ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52402 req_id = 521c4e32-a309-40ce-8a6c-822e13589105 response_code = 200 uri = /v2p
21592025-01-10T22:00:05.185ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49662
21602025-01-10T22:00:05.185ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
21612025-01-10T22:00:05.222ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
21622025-01-10T22:00:05.276ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90301 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:49662 req_id = 8e84f456-511d-4698-82a6-31989439c654 response_code = 204 uri = /switch-ports
21632025-01-10T22:00:05.276ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49662 req_id = 86ab4fb2-5b8a-4490-b3e9-27c7cad225d7 response_code = 200 uri = /network-bootstore-config
21642025-01-10T22:00:05.955ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58045
21652025-01-10T22:00:05.955ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58045 req_id = e9e650b0-72e1-4218-b413-98c8d23f7b75 response_code = 200 uri = /vpc-routes
21662025-01-10T22:00:06.022ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
21672025-01-10T22:00:06.022ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58045 req_id = cf1e2e1e-2ef7-4628-80f9-7107b6fead47 response_code = 204 uri = /eip-gateways
21682025-01-10T22:00:06.031ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 312 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58045 req_id = 6d04aa83-29f9-4a43-9b35-4b68a80dae4b response_code = 204 uri = /vpc-routes
21692025-01-10T22:00:06.092ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50507
21702025-01-10T22:00:06.092ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 141 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50507 req_id = e13f6d8a-db76-4bfb-8565-953494f61e9e response_code = 200 uri = /v2p
21712025-01-10T22:00:06.107ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62161
21722025-01-10T22:00:06.107ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
21732025-01-10T22:00:06.145ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
21742025-01-10T22:00:06.197ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90273 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:62161 req_id = 255d8f9b-27cb-45c3-ac67-5d4647b8aa52 response_code = 204 uri = /switch-ports
21752025-01-10T22:00:06.197ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51494
21762025-01-10T22:00:06.197ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62161 req_id = ea6fbc4a-e55b-423c-82db-d157982878e4 response_code = 200 uri = /network-bootstore-config
21772025-01-10T22:00:06.198ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51494 req_id = 2954fdda-8efd-4d1b-b6cd-f27bbae40767 response_code = 200 uri = /v2p
21782025-01-10T22:00:06.214ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60932
21792025-01-10T22:00:06.214ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 81 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60932 req_id = 1b957b48-80dd-48d6-8b77-11a248bebc9c response_code = 200 uri = /vpc-routes
21802025-01-10T22:00:06.227ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
21812025-01-10T22:00:06.227ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 80 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:60932 req_id = 9f84af75-ede7-41ed-88cd-cb4e789e9037 response_code = 204 uri = /eip-gateways
21822025-01-10T22:00:06.236ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 217 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:60932 req_id = b8efc674-db10-4c33-90b2-61edce849a23 response_code = 204 uri = /vpc-routes
21832025-01-10T22:00:06.292ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51408
21842025-01-10T22:00:06.292ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
21852025-01-10T22:00:06.330ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
21862025-01-10T22:00:06.382ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90012 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:51408 req_id = dd0056d8-9a3d-40a5-a7a8-19572c292bfd response_code = 204 uri = /switch-ports
21872025-01-10T22:00:06.383ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51408 req_id = 5861353c-adc3-493a-89eb-51276572acb1 response_code = 200 uri = /network-bootstore-config
21882025-01-10T22:00:17.690ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6797 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = cf25347d-e0f6-4c4f-8f3f-8973bce0e381 response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
21892025-01-10T22:00:25.030ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:59659
21902025-01-10T22:00:25.030ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 77 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:59659 req_id = 10ee86b4-bd10-43de-adcd-2941bb75c86d response_code = 200 uri = /baseboard
21912025-01-10T22:00:35.229ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59614
21922025-01-10T22:00:35.229ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59614 req_id = e64f665f-271d-4113-817d-3140c69fb241 response_code = 200 uri = /v2p
21932025-01-10T22:00:35.374ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41558
21942025-01-10T22:00:35.375ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 28 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41558 req_id = 2507fc14-1cae-4d1a-80f8-5eba0118bd94 response_code = 200 uri = /vpc-routes
21952025-01-10T22:00:35.436ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
21962025-01-10T22:00:35.436ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41558 req_id = c468b583-48a3-4b84-b70e-dd548979041e response_code = 204 uri = /eip-gateways
21972025-01-10T22:00:35.446ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 281 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41558 req_id = 70734ea9-8e00-41c2-bb30-55241330958c response_code = 204 uri = /vpc-routes
21982025-01-10T22:00:35.509ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36625
21992025-01-10T22:00:35.509ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
22002025-01-10T22:00:35.544ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
22012025-01-10T22:00:35.600ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90559 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:36625 req_id = d352621a-d214-418c-aa4c-35738dc74239 response_code = 204 uri = /switch-ports
22022025-01-10T22:00:35.600ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36625 req_id = b8e1f2de-91cd-45c6-ab40-d8d12c12defb response_code = 200 uri = /network-bootstore-config
22032025-01-10T22:00:36.056ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41846
22042025-01-10T22:00:36.056ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 43 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41846 req_id = e5e6f466-e6b5-42a3-a806-f848bd5356ac response_code = 200 uri = /vpc-routes
22052025-01-10T22:00:36.122ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
22062025-01-10T22:00:36.122ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:41846 req_id = abde3077-0f69-42f1-8ecb-a1550c75df88 response_code = 204 uri = /eip-gateways
22072025-01-10T22:00:36.182ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55777
22082025-01-10T22:00:36.182ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55777 req_id = e368529e-67c7-45d7-be18-86874e065928 response_code = 200 uri = /v2p
22092025-01-10T22:00:36.189ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 205 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:41846 req_id = 58bcc425-7ffc-477c-aa3e-66c711ecfc63 response_code = 204 uri = /vpc-routes
22102025-01-10T22:00:36.203ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47481
22112025-01-10T22:00:36.203ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 33 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47481 req_id = a38d674f-d6da-49f1-89f8-3977da8ac2fd response_code = 200 uri = /vpc-routes
22122025-01-10T22:00:36.230ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38695
22132025-01-10T22:00:36.231ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
22142025-01-10T22:00:36.268ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
22152025-01-10T22:00:36.323ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92535 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:38695 req_id = 697f06c7-0b8d-4ea8-87d0-a41b4d90ce47 response_code = 204 uri = /switch-ports
22162025-01-10T22:00:36.323ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
22172025-01-10T22:00:36.323ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 68 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47481 req_id = f35f10c6-c2df-4120-a02d-f8e251072fc7 response_code = 204 uri = /eip-gateways
22182025-01-10T22:00:36.323ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38695 req_id = b8258e16-ad91-4daf-96c1-d2d973b0836c response_code = 200 uri = /network-bootstore-config
22192025-01-10T22:00:36.377ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59059
22202025-01-10T22:00:36.377ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 200 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59059 req_id = ec9f5187-02b1-4a59-befd-52feaad3df06 response_code = 200 uri = /v2p
22212025-01-10T22:00:36.386ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 250 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47481 req_id = ae0434fb-bb26-41fb-8aa4-e069029d95b1 response_code = 204 uri = /vpc-routes
22222025-01-10T22:00:36.452ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47077
22232025-01-10T22:00:36.452ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
22242025-01-10T22:00:36.486ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
22252025-01-10T22:00:36.541ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89333 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:47077 req_id = d06e9121-c2f9-41f7-8502-060b2314cd5b response_code = 204 uri = /switch-ports
22262025-01-10T22:00:36.541ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47077 req_id = cc8f849c-a5db-4ab4-9e45-daa946a4f286 response_code = 200 uri = /network-bootstore-config
22272025-01-10T22:00:47.692ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8248 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = 05385d06-70f1-47d9-9ad6-0644cc8e1ebc response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
22282025-01-10T22:00:55.064ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:52526
22292025-01-10T22:00:55.064ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 28 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:52526 req_id = ac218dcb-8d1c-4c03-974b-900331e59e6f response_code = 200 uri = /baseboard
22302025-01-10T22:01:05.023ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61066
22312025-01-10T22:01:05.023ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61066 req_id = 1eaf261b-d1b3-4d31-baf6-a4c375e359b2 response_code = 200 uri = /vpc-routes
22322025-01-10T22:01:05.071ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
22332025-01-10T22:01:05.071ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61066 req_id = 622e4cdf-0eaa-4a83-9779-3fcafb64a03a response_code = 204 uri = /eip-gateways
22342025-01-10T22:01:05.081ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 193 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61066 req_id = 5dc924e4-d2aa-4ac8-933b-159683a0dc5b response_code = 204 uri = /vpc-routes
22352025-01-10T22:01:05.181ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60451
22362025-01-10T22:01:05.181ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60451 req_id = 4f9e7d8a-ae28-4cfb-8230-2b66eaad3d22 response_code = 200 uri = /v2p
22372025-01-10T22:01:05.197ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53838
22382025-01-10T22:01:05.198ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
22392025-01-10T22:01:05.236ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
22402025-01-10T22:01:05.293ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95482 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:53838 req_id = 695a648b-6a8f-40c2-9f18-f74760e462cb response_code = 204 uri = /switch-ports
22412025-01-10T22:01:05.293ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53838 req_id = b5665fe4-8a02-41f8-b91f-e45d6be0a0d6 response_code = 200 uri = /network-bootstore-config
22422025-01-10T22:01:05.902ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38115
22432025-01-10T22:01:05.902ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38115 req_id = c13a63fc-e433-46bd-a46a-2ebe7402cb23 response_code = 200 uri = /vpc-routes
22442025-01-10T22:01:05.959ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42799
22452025-01-10T22:01:05.959ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42799 req_id = 369f9907-40be-4a47-8e9d-561f35a1e2dd response_code = 200 uri = /v2p
22462025-01-10T22:01:05.965ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
22472025-01-10T22:01:05.965ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38115 req_id = dfaf769b-6526-4f2e-9e62-f70cfcf443ba response_code = 204 uri = /eip-gateways
22482025-01-10T22:01:05.976ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 311 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38115 req_id = f9311395-c849-4b52-b98e-cfac0b43be9e response_code = 204 uri = /vpc-routes
22492025-01-10T22:01:06.005ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34079
22502025-01-10T22:01:06.005ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
22512025-01-10T22:01:06.044ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
22522025-01-10T22:01:06.101ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95765 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:34079 req_id = c12e925b-6c8e-4835-82f2-20c29d27723b response_code = 204 uri = /switch-ports
22532025-01-10T22:01:06.101ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58130
22542025-01-10T22:01:06.101ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 230 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34079 req_id = 1a8a5fe7-a920-4dd3-82bc-2005df58d9f7 response_code = 200 uri = /network-bootstore-config
22552025-01-10T22:01:06.101ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58130 req_id = 7bb05c13-105d-4634-bbd2-cdb586d786d0 response_code = 200 uri = /vpc-routes
22562025-01-10T22:01:06.119ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
22572025-01-10T22:01:06.119ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 144 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58130 req_id = 40adf4f0-4f4c-4fc3-a9f4-f27d510c08de response_code = 204 uri = /eip-gateways
22582025-01-10T22:01:06.128ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 177 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58130 req_id = 5b916b6b-3056-4b58-821e-815e0c8dd474 response_code = 204 uri = /vpc-routes
22592025-01-10T22:01:06.151ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62229
22602025-01-10T22:01:06.151ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
22612025-01-10T22:01:06.188ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
22622025-01-10T22:01:06.244ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93257 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:62229 req_id = 302c5433-1ac7-4fdf-be2f-57ce28b85276 response_code = 204 uri = /switch-ports
22632025-01-10T22:01:06.245ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52703
22642025-01-10T22:01:06.245ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 180 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52703 req_id = 4d91d87d-2d44-479b-841f-0b5ab69ae13e response_code = 200 uri = /v2p
22652025-01-10T22:01:06.245ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 226 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62229 req_id = 3c22d6bd-bfd1-478f-a3dd-d59f4f3febe6 response_code = 200 uri = /network-bootstore-config
22662025-01-10T22:01:17.692ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8451 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = 09b7fd99-9433-4ccd-bd62-f58afe44dd85 response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
22672025-01-10T22:01:25.095ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:42064
22682025-01-10T22:01:25.095ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 32 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:42064 req_id = 11ff1ea8-534f-4cf3-b6ad-158aa8ef91c9 response_code = 200 uri = /baseboard
22692025-01-10T22:01:35.372ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60198
22702025-01-10T22:01:35.372ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60198 req_id = 6f9e6f70-db24-493c-8c51-8eb317cd2cc7 response_code = 200 uri = /vpc-routes
22712025-01-10T22:01:35.421ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
22722025-01-10T22:01:35.421ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60198 req_id = 013cfd06-d8c0-48dd-9002-debc2443c070 response_code = 204 uri = /eip-gateways
22732025-01-10T22:01:35.472ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36929
22742025-01-10T22:01:35.472ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 152 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36929 req_id = fba69fd5-42ef-4736-929c-7728750b2bb8 response_code = 200 uri = /v2p
22752025-01-10T22:01:35.475ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 186 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60198 req_id = a29d6428-9395-4ad3-af84-f7ec8bd43142 response_code = 204 uri = /vpc-routes
22762025-01-10T22:01:35.506ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47798
22772025-01-10T22:01:35.506ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
22782025-01-10T22:01:35.546ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
22792025-01-10T22:01:35.602ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96259 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:47798 req_id = d7ead4c0-0e3f-4a6b-8279-4fbeed61bbd8 response_code = 204 uri = /switch-ports
22802025-01-10T22:01:35.603ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47798 req_id = 05ffe403-5746-40e7-843e-7fbd0f8771ea response_code = 200 uri = /network-bootstore-config
22812025-01-10T22:01:35.655ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42317
22822025-01-10T22:01:35.656ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6decc476-dbff-4cb1-a909-d9d4408adaab, kind: Service { id: f44690fa-03a0-42b0-bb97-26c3d20aac30 }, name: Name("external-dns-f44690fa-03a0-42b0-bb97-26c3d20aac30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 143])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6f22ade2-da2d-4cbc-9141-adcc3b335aed, kind: Service { id: bce93894-738c-4596-b983-281da931e977 }, name: Name("external-dns-bce93894-738c-4596-b983-281da931e977"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 88])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 8448bae1-35b5-443a-8e61-6f2848603054, kind: Service { id: 4106fd29-fc2d-4538-b321-136cbc8ea9d2 }, name: Name("nexus-4106fd29-fc2d-4538-b321-136cbc8ea9d2"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 27])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9676df2d-b2c5-40f4-ae20-30ad4de6c8e2, kind: Service { id: 19558293-ba4c-462e-8746-8d928c3a9612 }, name: Name("nexus-19558293-ba4c-462e-8746-8d928c3a9612"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 42])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bf22f522-6412-4896-90b5-9bc8aab0192d, kind: Service { id: b507bc15-7e7b-4d56-a806-1da438b76052 }, name: Name("nexus-b507bc15-7e7b-4d56-a806-1da438b76052"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 137])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
22832025-01-10T22:01:35.656ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
22842025-01-10T22:01:35.656ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
22852025-01-10T22:01:35.656ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22862025-01-10T22:01:35.656ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22872025-01-10T22:01:35.656ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
22882025-01-10T22:01:35.656ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22892025-01-10T22:01:35.656ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 469 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:42317 req_id = d4caf9be-feab-400d-99cb-8e0463804a2c response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
22902025-01-10T22:01:36.036ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38155
22912025-01-10T22:01:36.036ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38155 req_id = 69b6fef4-b971-4f78-bfb4-e4dfd0488408 response_code = 200 uri = /vpc-routes
22922025-01-10T22:01:36.085ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
22932025-01-10T22:01:36.085ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38155 req_id = 352371ce-ce4b-48a1-842a-70e1013b3854 response_code = 204 uri = /eip-gateways
22942025-01-10T22:01:36.157ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55173
22952025-01-10T22:01:36.157ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55173 req_id = 8d22e53a-066a-4069-8d76-5d451d926288 response_code = 200 uri = /v2p
22962025-01-10T22:01:36.164ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 255 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38155 req_id = 7a46d61b-2380-4f1c-a089-a28898c35545 response_code = 204 uri = /vpc-routes
22972025-01-10T22:01:36.197ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41614
22982025-01-10T22:01:36.197ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
22992025-01-10T22:01:36.235ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
23002025-01-10T22:01:36.293ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96109 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:41614 req_id = 744c55fd-3e52-4d55-bad5-6dd216046cce response_code = 204 uri = /switch-ports
23012025-01-10T22:01:36.293ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39475
23022025-01-10T22:01:36.293ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39475 req_id = 27aaff13-5b2b-4083-8b97-d4d27f22aaff response_code = 200 uri = /vpc-routes
23032025-01-10T22:01:36.293ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41614 req_id = eb44cd1d-50d7-4f08-a228-8b9dcfb92f56 response_code = 200 uri = /network-bootstore-config
23042025-01-10T22:01:36.318ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
23052025-01-10T22:01:36.318ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:39475 req_id = 32528f94-e86c-4636-b8dc-2c0791e06265 response_code = 204 uri = /eip-gateways
23062025-01-10T22:01:36.334ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 244 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:39475 req_id = 2be082d9-e5e9-474d-8feb-aad47c01d99d response_code = 204 uri = /vpc-routes
23072025-01-10T22:01:36.445ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42480
23082025-01-10T22:01:36.446ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6decc476-dbff-4cb1-a909-d9d4408adaab, kind: Service { id: f44690fa-03a0-42b0-bb97-26c3d20aac30 }, name: Name("external-dns-f44690fa-03a0-42b0-bb97-26c3d20aac30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 143])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6f22ade2-da2d-4cbc-9141-adcc3b335aed, kind: Service { id: bce93894-738c-4596-b983-281da931e977 }, name: Name("external-dns-bce93894-738c-4596-b983-281da931e977"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 88])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 8448bae1-35b5-443a-8e61-6f2848603054, kind: Service { id: 4106fd29-fc2d-4538-b321-136cbc8ea9d2 }, name: Name("nexus-4106fd29-fc2d-4538-b321-136cbc8ea9d2"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 27])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9676df2d-b2c5-40f4-ae20-30ad4de6c8e2, kind: Service { id: 19558293-ba4c-462e-8746-8d928c3a9612 }, name: Name("nexus-19558293-ba4c-462e-8746-8d928c3a9612"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 42])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bf22f522-6412-4896-90b5-9bc8aab0192d, kind: Service { id: b507bc15-7e7b-4d56-a806-1da438b76052 }, name: Name("nexus-b507bc15-7e7b-4d56-a806-1da438b76052"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 137])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
23092025-01-10T22:01:36.446ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23102025-01-10T22:01:36.446ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23112025-01-10T22:01:36.446ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23122025-01-10T22:01:36.446ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23132025-01-10T22:01:36.446ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
23142025-01-10T22:01:36.446ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23152025-01-10T22:01:36.446ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 358 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42480 req_id = ece28d71-9cc3-436f-b7ad-37e60dda8d5e response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23162025-01-10T22:01:36.507ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63081
23172025-01-10T22:01:36.507ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
23182025-01-10T22:01:36.543ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
23192025-01-10T22:01:36.598ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90687 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:63081 req_id = b605b629-7beb-4442-81ca-9a41501b7326 response_code = 204 uri = /switch-ports
23202025-01-10T22:01:36.598ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41750
23212025-01-10T22:01:36.598ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63081 req_id = e98b7d81-fbff-451f-97ae-c1183ee78b65 response_code = 200 uri = /network-bootstore-config
23222025-01-10T22:01:36.598ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 240 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41750 req_id = 65af9a81-9a36-48c6-90f6-4b11cee59066 response_code = 200 uri = /v2p
23232025-01-10T22:01:36.627ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43033
23242025-01-10T22:01:36.627ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6decc476-dbff-4cb1-a909-d9d4408adaab, kind: Service { id: f44690fa-03a0-42b0-bb97-26c3d20aac30 }, name: Name("external-dns-f44690fa-03a0-42b0-bb97-26c3d20aac30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 143])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6f22ade2-da2d-4cbc-9141-adcc3b335aed, kind: Service { id: bce93894-738c-4596-b983-281da931e977 }, name: Name("external-dns-bce93894-738c-4596-b983-281da931e977"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 88])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 8448bae1-35b5-443a-8e61-6f2848603054, kind: Service { id: 4106fd29-fc2d-4538-b321-136cbc8ea9d2 }, name: Name("nexus-4106fd29-fc2d-4538-b321-136cbc8ea9d2"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 27])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9676df2d-b2c5-40f4-ae20-30ad4de6c8e2, kind: Service { id: 19558293-ba4c-462e-8746-8d928c3a9612 }, name: Name("nexus-19558293-ba4c-462e-8746-8d928c3a9612"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 42])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bf22f522-6412-4896-90b5-9bc8aab0192d, kind: Service { id: b507bc15-7e7b-4d56-a806-1da438b76052 }, name: Name("nexus-b507bc15-7e7b-4d56-a806-1da438b76052"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 137])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
23252025-01-10T22:01:36.627ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23262025-01-10T22:01:36.627ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23272025-01-10T22:01:36.627ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23282025-01-10T22:01:36.627ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23292025-01-10T22:01:36.627ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
23302025-01-10T22:01:36.627ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23312025-01-10T22:01:36.627ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 406 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43033 req_id = 920bdd38-fdf8-4922-9527-edf7e0998a0e response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23322025-01-10T22:01:47.690ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7048 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = 034cfeb2-4b7a-4c4b-897d-c11b7f59259c response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
23332025-01-10T22:01:55.129ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:34991
23342025-01-10T22:01:55.129ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 85 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:34991 req_id = 6e2638ee-35cb-433e-9b8b-b5118b90328f response_code = 200 uri = /baseboard
23352025-01-10T22:02:05.058ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35206
23362025-01-10T22:02:05.058ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35206 req_id = 3e964dd1-9130-485e-b688-69a07a36a3b3 response_code = 200 uri = /vpc-routes
23372025-01-10T22:02:05.105ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
23382025-01-10T22:02:05.105ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35206 req_id = 82367e54-1d9f-456e-aaeb-dde761238904 response_code = 204 uri = /eip-gateways
23392025-01-10T22:02:05.115ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 191 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35206 req_id = 2e5032ce-f5a8-4f11-a768-046a38759ac5 response_code = 204 uri = /vpc-routes
23402025-01-10T22:02:05.125ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59127
23412025-01-10T22:02:05.125ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59127 req_id = b1220089-db4e-4713-b479-c9c1a1e7249a response_code = 200 uri = /v2p
23422025-01-10T22:02:05.179ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51805
23432025-01-10T22:02:05.179ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
23442025-01-10T22:02:05.215ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
23452025-01-10T22:02:05.268ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89935 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:51805 req_id = b2726870-2504-4cde-ba0e-66ceef99fc99 response_code = 204 uri = /switch-ports
23462025-01-10T22:02:05.269ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51805 req_id = 2d557830-d0f5-42cc-9fcd-5ecd15529166 response_code = 200 uri = /network-bootstore-config
23472025-01-10T22:02:05.852ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42075
23482025-01-10T22:02:05.852ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42075 req_id = 8622ccf7-3bf1-4433-acb7-884b7dcc924c response_code = 200 uri = /vpc-routes
23492025-01-10T22:02:05.912ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
23502025-01-10T22:02:05.912ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42075 req_id = 6e8daf39-a316-46f9-b4f6-c5c88977ef30 response_code = 204 uri = /eip-gateways
23512025-01-10T22:02:05.922ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 248 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42075 req_id = 2a241e3e-6b61-40a0-baf5-bc72c751bd78 response_code = 204 uri = /vpc-routes
23522025-01-10T22:02:06.048ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44122
23532025-01-10T22:02:06.049ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44122 req_id = f304a854-3ec7-4459-ad0c-15e5d7764ed6 response_code = 200 uri = /v2p
23542025-01-10T22:02:06.103ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54865
23552025-01-10T22:02:06.103ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
23562025-01-10T22:02:06.141ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
23572025-01-10T22:02:06.197ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93416 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:54865 req_id = 677b07e1-4d80-4b26-97dc-fb6228928c2c response_code = 204 uri = /switch-ports
23582025-01-10T22:02:06.197ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58032
23592025-01-10T22:02:06.197ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58032 req_id = 79626e64-68dd-476c-83a2-da04f4ea82ba response_code = 200 uri = /vpc-routes
23602025-01-10T22:02:06.197ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 206 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54865 req_id = 7ea0d1f2-675e-448e-8a16-636c7dbec914 response_code = 200 uri = /network-bootstore-config
23612025-01-10T22:02:06.238ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
23622025-01-10T22:02:06.238ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58032 req_id = 7aa1e86a-67e5-40de-881a-de264bb430f1 response_code = 204 uri = /eip-gateways
23632025-01-10T22:02:06.247ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57155
23642025-01-10T22:02:06.247ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
23652025-01-10T22:02:06.287ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
23662025-01-10T22:02:06.340ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93300 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:57155 req_id = 75fa30ad-3f95-4b0e-8814-740b56553ffd response_code = 204 uri = /switch-ports
23672025-01-10T22:02:06.340ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 266 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58032 req_id = 0d21e217-2596-4bb3-980b-06bab231d19a response_code = 204 uri = /vpc-routes
23682025-01-10T22:02:06.341ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57155 req_id = 295a79a7-6791-4b77-8821-f841f2a19727 response_code = 200 uri = /network-bootstore-config
23692025-01-10T22:02:06.353ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33375
23702025-01-10T22:02:06.353ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 150 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33375 req_id = cec81b6a-e8ca-4336-9527-2ea471c85f34 response_code = 200 uri = /v2p
23712025-01-10T22:02:17.690ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6857 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = 1b0949c3-7ae8-42b6-a85f-ee7f7de0f186 response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
23722025-01-10T22:02:25.162ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47029
23732025-01-10T22:02:25.162ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 27 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47029 req_id = ba9e4400-12c3-43f7-a4d3-82193997b65b response_code = 200 uri = /baseboard
23742025-01-10T22:02:35.174ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40811
23752025-01-10T22:02:35.175ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40811 req_id = 2a3bb6dd-f3db-4d0a-aceb-3dc0e6974c4d response_code = 200 uri = /vpc-routes
23762025-01-10T22:02:35.191ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
23772025-01-10T22:02:35.191ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:40811 req_id = ea549432-bc84-4e41-a077-8dcb54327cb2 response_code = 204 uri = /eip-gateways
23782025-01-10T22:02:35.201ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 246 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:40811 req_id = bf60ef7e-2296-4cd0-8c95-151a3f4df6b7 response_code = 204 uri = /vpc-routes
23792025-01-10T22:02:35.333ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57570
23802025-01-10T22:02:35.334ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57570 req_id = 06be10ca-37c5-4498-b069-97fd14799af9 response_code = 200 uri = /v2p
23812025-01-10T22:02:35.386ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45290
23822025-01-10T22:02:35.386ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
23832025-01-10T22:02:35.424ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
23842025-01-10T22:02:35.478ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91284 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:45290 req_id = 4b03a355-5e3a-4c4d-b152-b26d713fae52 response_code = 204 uri = /switch-ports
23852025-01-10T22:02:35.478ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45290 req_id = 2b2b3b50-5c81-4ab4-a959-d86f93e5a943 response_code = 200 uri = /network-bootstore-config
23862025-01-10T22:02:36.006ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57511
23872025-01-10T22:02:36.006ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57511 req_id = 4b3f5c57-79ed-4e04-b743-54650496a070 response_code = 200 uri = /vpc-routes
23882025-01-10T22:02:36.080ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
23892025-01-10T22:02:36.080ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57511 req_id = 672f0b6e-a369-4df4-8d65-d76da60d1a84 response_code = 204 uri = /eip-gateways
23902025-01-10T22:02:36.090ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 222 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57511 req_id = b4373d0c-f2af-43ca-b4b0-441e41e1d05d response_code = 204 uri = /vpc-routes
23912025-01-10T22:02:36.283ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57060
23922025-01-10T22:02:36.283ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57060 req_id = 316a41fe-74bd-4f82-80a5-3893da941d53 response_code = 200 uri = /vpc-routes
23932025-01-10T22:02:36.287ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35338
23942025-01-10T22:02:36.287ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
23952025-01-10T22:02:36.326ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
23962025-01-10T22:02:36.382ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94247 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:35338 req_id = cf701c6f-660a-4a06-be26-a113023d8bf5 response_code = 204 uri = /switch-ports
23972025-01-10T22:02:36.382ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36976
23982025-01-10T22:02:36.382ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 189 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35338 req_id = 60f1c828-4296-4986-b484-9adef6152eb4 response_code = 200 uri = /network-bootstore-config
23992025-01-10T22:02:36.382ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
24002025-01-10T22:02:36.382ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 202 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57060 req_id = 99ef680d-efd1-4638-8f2a-0e8165d4ab70 response_code = 204 uri = /eip-gateways
24012025-01-10T22:02:36.382ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 227 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36976 req_id = 9ca670b0-8de5-4e9a-9868-243554dec9ad response_code = 200 uri = /v2p
24022025-01-10T22:02:36.393ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 325 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57060 req_id = 86dc0cdd-4a09-4d53-ae10-3814608b8900 response_code = 204 uri = /vpc-routes
24032025-01-10T22:02:36.435ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46591
24042025-01-10T22:02:36.435ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46591 req_id = 55730828-84ae-4763-acaf-cf377b7967e2 response_code = 200 uri = /v2p
24052025-01-10T22:02:36.526ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53217
24062025-01-10T22:02:36.526ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
24072025-01-10T22:02:36.561ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
24082025-01-10T22:02:36.615ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89283 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:53217 req_id = 16380028-cb18-4e34-9f91-93292d920436 response_code = 204 uri = /switch-ports
24092025-01-10T22:02:36.616ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53217 req_id = a0c884bb-20b3-473b-b722-960460999fc5 response_code = 200 uri = /network-bootstore-config
24102025-01-10T22:02:47.689ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6941 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = 928b947a-8595-4c9e-b168-89d5dfdb26ca response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
24112025-01-10T22:02:55.196ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:46694
24122025-01-10T22:02:55.196ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:46694 req_id = d5fe1f07-f63b-4dd0-b8c8-2a2aebf46355 response_code = 200 uri = /baseboard
24132025-01-10T22:03:05.004ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59591
24142025-01-10T22:03:05.004ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59591 req_id = 7be56732-a85c-429c-8ff9-0f558aa3ac07 response_code = 200 uri = /vpc-routes
24152025-01-10T22:03:05.055ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
24162025-01-10T22:03:05.055ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 77 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:59591 req_id = 2740cbc2-7e6c-4628-a607-e4d1910ae5a0 response_code = 204 uri = /eip-gateways
24172025-01-10T22:03:05.115ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52983
24182025-01-10T22:03:05.115ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52983 req_id = 48f3ed87-cfdb-4954-bfda-33e0dcfc9ffd response_code = 200 uri = /v2p
24192025-01-10T22:03:05.118ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 184 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:59591 req_id = 24d6146a-6e0c-4e33-a833-16bb27e876bd response_code = 204 uri = /vpc-routes
24202025-01-10T22:03:05.157ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47437
24212025-01-10T22:03:05.157ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
24222025-01-10T22:03:05.191ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
24232025-01-10T22:03:05.244ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86989 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:47437 req_id = 3fd21450-41bc-48ce-ac95-f7eb54bb7caf response_code = 204 uri = /switch-ports
24242025-01-10T22:03:05.244ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 157 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47437 req_id = 5397011a-54f3-4d60-8820-0acf39cfcb13 response_code = 200 uri = /network-bootstore-config
24252025-01-10T22:03:05.961ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43147
24262025-01-10T22:03:05.961ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43147 req_id = c5564f92-ecfe-4a01-86a5-2a316361cf4c response_code = 200 uri = /vpc-routes
24272025-01-10T22:03:05.982ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55634
24282025-01-10T22:03:05.982ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55634 req_id = 045587d3-9a95-4cff-a337-2b18346186e6 response_code = 200 uri = /v2p
24292025-01-10T22:03:05.986ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
24302025-01-10T22:03:05.986ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 68 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:43147 req_id = 845018bf-da2d-4d0a-8173-54bce3836f2e response_code = 204 uri = /eip-gateways
24312025-01-10T22:03:05.998ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 201 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:43147 req_id = f0d3b04b-b96a-4ba7-a606-c69827060fbf response_code = 204 uri = /vpc-routes
24322025-01-10T22:03:06.043ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34204
24332025-01-10T22:03:06.044ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
24342025-01-10T22:03:06.083ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
24352025-01-10T22:03:06.141ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97153 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:34204 req_id = e9293df3-0373-4cfc-a5be-e61e2f668143 response_code = 204 uri = /switch-ports
24362025-01-10T22:03:06.141ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36562
24372025-01-10T22:03:06.141ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36562 req_id = 9fac8b23-675f-418d-a88d-5c1133c959b6 response_code = 200 uri = /vpc-routes
24382025-01-10T22:03:06.141ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34204 req_id = 4e002c74-45c0-4bba-9b8b-f08d5026f48e response_code = 200 uri = /network-bootstore-config
24392025-01-10T22:03:06.156ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34471
24402025-01-10T22:03:06.156ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 214 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34471 req_id = 5db3f680-0c1d-45ef-8b2c-ca136a37f1f7 response_code = 200 uri = /v2p
24412025-01-10T22:03:06.171ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
24422025-01-10T22:03:06.171ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:36562 req_id = fae498a2-2d95-4ce3-b582-63e9c9ea80c6 response_code = 204 uri = /eip-gateways
24432025-01-10T22:03:06.179ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 345 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:36562 req_id = 4f4eb878-0273-467d-bf5d-1c60bdbbd5f9 response_code = 204 uri = /vpc-routes
24442025-01-10T22:03:06.224ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56935
24452025-01-10T22:03:06.225ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
24462025-01-10T22:03:06.262ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
24472025-01-10T22:03:06.315ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90445 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:56935 req_id = d81685db-73eb-4ec8-9074-e296ac2118ff response_code = 204 uri = /switch-ports
24482025-01-10T22:03:06.315ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56935 req_id = ce031af8-ea01-4564-b700-3a0bb82285b7 response_code = 200 uri = /network-bootstore-config
24492025-01-10T22:03:17.690ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7171 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = 308f28fa-4f15-43dc-842f-3f350bfc8d85 response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
24502025-01-10T22:03:25.228ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:38227
24512025-01-10T22:03:25.228ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:38227 req_id = 7e98d7b9-9180-4720-bb8b-8d8075cf312f response_code = 200 uri = /baseboard
24522025-01-10T22:03:35.177ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55581
24532025-01-10T22:03:35.177ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55581 req_id = b51e0b52-0239-40b2-9df6-d82d3bcff9d6 response_code = 200 uri = /vpc-routes
24542025-01-10T22:03:35.229ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
24552025-01-10T22:03:35.229ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55581 req_id = 32454ead-5d33-4369-818c-e1425ed19036 response_code = 204 uri = /eip-gateways
24562025-01-10T22:03:35.238ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 230 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:55581 req_id = 4f2332fb-7a72-40b8-a43e-345e8f3f1a32 response_code = 204 uri = /vpc-routes
24572025-01-10T22:03:35.281ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39834
24582025-01-10T22:03:35.282ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
24592025-01-10T22:03:35.321ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
24602025-01-10T22:03:35.376ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94562 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:39834 req_id = 2dda2210-14ed-434b-b62f-27dd686a577f response_code = 204 uri = /switch-ports
24612025-01-10T22:03:35.376ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39834 req_id = f198730f-7c81-48bf-8d8e-e318c03ef562 response_code = 200 uri = /network-bootstore-config
24622025-01-10T22:03:35.396ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49134
24632025-01-10T22:03:35.396ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 155 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49134 req_id = 77f48040-811d-4e22-b367-51f1194b30a8 response_code = 200 uri = /v2p
24642025-01-10T22:03:35.998ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39767
24652025-01-10T22:03:35.998ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39767 req_id = bc41af43-e9b2-4992-b3a6-5c5f404bdb68 response_code = 200 uri = /vpc-routes
24662025-01-10T22:03:36.026ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
24672025-01-10T22:03:36.026ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39767 req_id = 7a8646aa-b5d8-49dd-bede-9eef31ac4dde response_code = 204 uri = /eip-gateways
24682025-01-10T22:03:36.044ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 237 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39767 req_id = cc2c5bf5-5f14-4beb-9b64-f95ef1665925 response_code = 204 uri = /vpc-routes
24692025-01-10T22:03:36.241ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56328
24702025-01-10T22:03:36.241ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56328 req_id = 0c7590f6-b3a9-4416-bd9e-a675b4891c24 response_code = 200 uri = /vpc-routes
24712025-01-10T22:03:36.266ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62076
24722025-01-10T22:03:36.266ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 151 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62076 req_id = 22c3f9e1-09a8-494a-a66e-f67a1e3ca3d2 response_code = 200 uri = /v2p
24732025-01-10T22:03:36.294ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
24742025-01-10T22:03:36.294ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:56328 req_id = 9bcd248e-ac79-418c-905e-3fc0686f9788 response_code = 204 uri = /eip-gateways
24752025-01-10T22:03:36.303ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58675
24762025-01-10T22:03:36.303ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
24772025-01-10T22:03:36.341ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
24782025-01-10T22:03:36.391ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88222 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:58675 req_id = 80267a1f-434c-49b8-a46f-e7a5c3f35925 response_code = 204 uri = /switch-ports
24792025-01-10T22:03:36.391ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58675 req_id = c95af474-62f9-428b-9cfe-285a8dc10f63 response_code = 200 uri = /network-bootstore-config
24802025-01-10T22:03:36.391ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 365 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:56328 req_id = cb39e395-7069-40ee-9fef-4e935d80ab25 response_code = 204 uri = /vpc-routes
24812025-01-10T22:03:36.541ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43727
24822025-01-10T22:03:36.541ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43727 req_id = 88d57f12-623f-4a27-bb97-d89715b41651 response_code = 200 uri = /v2p
24832025-01-10T22:03:36.590ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36931
24842025-01-10T22:03:36.590ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
24852025-01-10T22:03:36.628ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
24862025-01-10T22:03:36.684ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93956 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:36931 req_id = 910582ca-fb5e-49d7-a99f-7e50e82aabfb response_code = 204 uri = /switch-ports
24872025-01-10T22:03:36.684ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36931 req_id = e373292a-418f-438d-b454-61d0d9ecc674 response_code = 200 uri = /network-bootstore-config
24882025-01-10T22:03:47.689ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7117 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = 2e64b59d-2df9-46ff-a223-aaa315f78a46 response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
24892025-01-10T22:03:55.260ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:56577
24902025-01-10T22:03:55.260ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 30 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:56577 req_id = 6f597285-14fc-4b4d-aa4b-ca156fc9ee14 response_code = 200 uri = /baseboard
24912025-01-10T22:04:05.057ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56965
24922025-01-10T22:04:05.057ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56965 req_id = 4d54bc95-4512-4c0f-985b-6767181e7446 response_code = 200 uri = /vpc-routes
24932025-01-10T22:04:05.103ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
24942025-01-10T22:04:05.103ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56965 req_id = 9f97a627-e05b-4383-9457-2d8fb1509981 response_code = 204 uri = /eip-gateways
24952025-01-10T22:04:05.112ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 276 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56965 req_id = 4bc88218-36d4-4e42-8afc-565d9c0fa644 response_code = 204 uri = /vpc-routes
24962025-01-10T22:04:05.226ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45231
24972025-01-10T22:04:05.226ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45231 req_id = 763a2261-e7ad-4a15-8ae0-9912c1294f5e response_code = 200 uri = /v2p
24982025-01-10T22:04:05.248ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44902
24992025-01-10T22:04:05.248ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
25002025-01-10T22:04:05.285ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
25012025-01-10T22:04:05.336ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88362 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:44902 req_id = 938f18d3-b8c8-4a6a-b4fa-b5f39ca1c50e response_code = 204 uri = /switch-ports
25022025-01-10T22:04:05.340ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44902 req_id = ff7f139f-86ab-40e9-b882-31ba8842608a response_code = 200 uri = /network-bootstore-config
25032025-01-10T22:04:05.937ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51198
25042025-01-10T22:04:05.938ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51198 req_id = 53723833-e0b6-4c37-96b4-4691466e560f response_code = 200 uri = /vpc-routes
25052025-01-10T22:04:05.962ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
25062025-01-10T22:04:05.962ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51198 req_id = cc94ea44-84b1-4374-9f4e-43c2df424d06 response_code = 204 uri = /eip-gateways
25072025-01-10T22:04:06.002ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44291
25082025-01-10T22:04:06.002ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 183 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44291 req_id = f97619c5-f30b-4ac8-aac6-bf8ab924230a response_code = 200 uri = /v2p
25092025-01-10T22:04:06.006ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 327 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51198 req_id = 0f64515e-bb21-4afe-a3fd-f9b27770ae74 response_code = 204 uri = /vpc-routes
25102025-01-10T22:04:06.113ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48785
25112025-01-10T22:04:06.113ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48785 req_id = 2cc29d95-511c-42a0-8bc8-5e8428d1ead8 response_code = 200 uri = /vpc-routes
25122025-01-10T22:04:06.120ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34088
25132025-01-10T22:04:06.121ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
25142025-01-10T22:04:06.159ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
25152025-01-10T22:04:06.210ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89330 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:34088 req_id = 74657276-ee33-4132-bcd8-f89be46effcf response_code = 204 uri = /switch-ports
25162025-01-10T22:04:06.210ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
25172025-01-10T22:04:06.210ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 141 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34088 req_id = 378fa42c-24ba-4b32-bf1d-ca8e3642e90b response_code = 200 uri = /network-bootstore-config
25182025-01-10T22:04:06.210ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 180 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48785 req_id = f00b3c1e-7d7b-4c6a-91c5-e0058bd7e797 response_code = 204 uri = /eip-gateways
25192025-01-10T22:04:06.219ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 262 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48785 req_id = 2365a166-19aa-433c-89e2-8e1bd6c6e051 response_code = 204 uri = /vpc-routes
25202025-01-10T22:04:06.231ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51454
25212025-01-10T22:04:06.231ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
25222025-01-10T22:04:06.268ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
25232025-01-10T22:04:06.309ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42145
25242025-01-10T22:04:06.310ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 179 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42145 req_id = a016c8e7-132c-4d0d-9daa-26d3799e55e2 response_code = 200 uri = /v2p
25252025-01-10T22:04:06.323ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91892 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:51454 req_id = e3d3231a-f43d-40d5-a0c5-5b6e62257fc3 response_code = 204 uri = /switch-ports
25262025-01-10T22:04:06.323ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51454 req_id = fd0db9b3-6d5c-4494-9ff0-81358cb109fc response_code = 200 uri = /network-bootstore-config
25272025-01-10T22:04:17.689ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7088 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = 5024af26-f64b-47cb-b267-fb5465200da0 response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
25282025-01-10T22:04:25.291ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:48955
25292025-01-10T22:04:25.291ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:48955 req_id = 2624daca-52ce-4d4c-a180-b925f06fcd3e response_code = 200 uri = /baseboard
25302025-01-10T22:04:28.750ZINFOSledAgent (auto-cleanup-task): running automatic periodic zone bundle cleanup file = sled-agent/src/zone_bundle.rs:197
25312025-01-10T22:04:28.750ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
25322025-01-10T22:04:28.780ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 3733, tv_nsec: 791922717 } time_to_next_cleanup = 599.99999522s
25332025-01-10T22:04:35.174ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49170
25342025-01-10T22:04:35.174ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49170 req_id = d69371b3-6613-4572-9550-847d6fb009fa response_code = 200 uri = /vpc-routes
25352025-01-10T22:04:35.225ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
25362025-01-10T22:04:35.225ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:49170 req_id = 6031a59e-383a-4fda-9978-dab33d37a437 response_code = 204 uri = /eip-gateways
25372025-01-10T22:04:35.234ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 184 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:49170 req_id = 1ee754fb-4d67-41ab-bfad-34988bd18f1c response_code = 204 uri = /vpc-routes
25382025-01-10T22:04:35.338ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36760
25392025-01-10T22:04:35.338ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36760 req_id = 01c8f169-3f17-4e58-96fc-ba2c35021895 response_code = 200 uri = /v2p
25402025-01-10T22:04:35.364ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41361
25412025-01-10T22:04:35.364ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
25422025-01-10T22:04:35.401ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
25432025-01-10T22:04:35.458ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93203 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:41361 req_id = ea488670-b6d7-400d-aacd-83ba49768b11 response_code = 204 uri = /switch-ports
25442025-01-10T22:04:35.458ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41361 req_id = 42441186-6913-47b9-8c9f-189f63f3f953 response_code = 200 uri = /network-bootstore-config
25452025-01-10T22:04:36.079ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44004
25462025-01-10T22:04:36.079ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44004 req_id = 06906760-d2ac-4d02-bf73-906c67300b1b response_code = 200 uri = /vpc-routes
25472025-01-10T22:04:36.083ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53906
25482025-01-10T22:04:36.083ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53906 req_id = c199622f-fb71-414e-b7ca-166b7a0d0053 response_code = 200 uri = /v2p
25492025-01-10T22:04:36.121ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
25502025-01-10T22:04:36.121ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44004 req_id = a3abb75f-7119-473b-82f3-48da120746d7 response_code = 204 uri = /eip-gateways
25512025-01-10T22:04:36.130ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 301 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44004 req_id = cd89d4bc-269c-47fe-8665-e87c4494b426 response_code = 204 uri = /vpc-routes
25522025-01-10T22:04:36.217ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56251
25532025-01-10T22:04:36.217ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56251 req_id = 14dcd11c-7334-4401-895b-3c5a8371d431 response_code = 200 uri = /vpc-routes
25542025-01-10T22:04:36.264ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
25552025-01-10T22:04:36.264ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:56251 req_id = 002dd8b7-b9ac-4b88-9911-4e297fe64be0 response_code = 204 uri = /eip-gateways
25562025-01-10T22:04:36.268ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56439
25572025-01-10T22:04:36.268ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 140 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56439 req_id = f9b10791-2c55-476c-9471-fdd17382fc69 response_code = 200 uri = /v2p
25582025-01-10T22:04:36.273ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 202 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:56251 req_id = 16cc2b69-9648-4fbd-9d12-dd5d56d4bd43 response_code = 204 uri = /vpc-routes
25592025-01-10T22:04:36.274ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54788
25602025-01-10T22:04:36.274ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
25612025-01-10T22:04:36.311ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
25622025-01-10T22:04:36.364ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90264 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:54788 req_id = 31a17c18-783c-4f4f-8e32-d18e968f40f9 response_code = 204 uri = /switch-ports
25632025-01-10T22:04:36.364ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49453
25642025-01-10T22:04:36.365ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
25652025-01-10T22:04:36.365ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 239 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54788 req_id = 38218458-402f-48cf-8dde-2983deccaf05 response_code = 200 uri = /network-bootstore-config
25662025-01-10T22:04:36.398ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
25672025-01-10T22:04:36.452ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87323 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:49453 req_id = a108cdf5-9f21-4c48-b104-ba3f528ba779 response_code = 204 uri = /switch-ports
25682025-01-10T22:04:36.452ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49453 req_id = e2456fad-9f1e-42bd-ad09-bb7f9f79b256 response_code = 200 uri = /network-bootstore-config
25692025-01-10T22:04:42.958ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
25702025-01-10T22:04:47.689ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7650 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = 28c132e0-1ee5-4430-ab6a-3aa7ee286605 response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
25712025-01-10T22:04:55.324ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:41517
25722025-01-10T22:04:55.324ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 28 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:41517 req_id = 934e3c4d-21a7-4ed4-a2ee-8e4a8d9a6914 response_code = 200 uri = /baseboard
25732025-01-10T22:05:05.026ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43449
25742025-01-10T22:05:05.026ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43449 req_id = c060c61b-399b-4043-80f2-2f6f320af143 response_code = 200 uri = /vpc-routes
25752025-01-10T22:05:05.041ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
25762025-01-10T22:05:05.041ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43449 req_id = 4f131c14-d508-47e0-85ae-10a5b1f25ab9 response_code = 204 uri = /eip-gateways
25772025-01-10T22:05:05.050ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43449 req_id = f50f678f-f7bc-46c1-81b1-441e47ba57c5 response_code = 204 uri = /vpc-routes
25782025-01-10T22:05:05.093ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60413
25792025-01-10T22:05:05.093ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
25802025-01-10T22:05:05.131ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
25812025-01-10T22:05:05.188ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94904 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:60413 req_id = 66b7ceae-8c6d-44c9-b9b1-8452f709f7ab response_code = 204 uri = /switch-ports
25822025-01-10T22:05:05.188ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43037
25832025-01-10T22:05:05.188ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 160 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43037 req_id = 470a3cb4-59a6-4336-b32d-fc871eadcfe1 response_code = 200 uri = /v2p
25842025-01-10T22:05:05.188ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 169 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60413 req_id = fc1e206e-4c33-41c5-9cad-53d998a5e551 response_code = 200 uri = /network-bootstore-config
25852025-01-10T22:05:05.907ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46916
25862025-01-10T22:05:05.907ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46916 req_id = abb4c2d3-6471-4520-ba64-996edcbe1e6a response_code = 200 uri = /vpc-routes
25872025-01-10T22:05:05.966ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
25882025-01-10T22:05:05.966ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46916 req_id = 743d2c46-c70c-4feb-931e-3a7c932b0ae6 response_code = 204 uri = /eip-gateways
25892025-01-10T22:05:05.980ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 235 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46916 req_id = 9f20a0fd-19b7-4d9d-a2ae-667dd70c374d response_code = 204 uri = /vpc-routes
25902025-01-10T22:05:05.986ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45363
25912025-01-10T22:05:05.986ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
25922025-01-10T22:05:06.028ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
25932025-01-10T22:05:06.094ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108125 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:45363 req_id = 68c45cde-ccb0-4cb1-b41c-d4fd515f93eb response_code = 204 uri = /switch-ports
25942025-01-10T22:05:06.094ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36071
25952025-01-10T22:05:06.095ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 265 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45363 req_id = 8e382919-6a43-4182-bd00-1a365e67a722 response_code = 200 uri = /network-bootstore-config
25962025-01-10T22:05:06.095ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 185 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36071 req_id = a5082e46-bc20-49db-b80d-9a166b5e7f3b response_code = 200 uri = /v2p
25972025-01-10T22:05:06.126ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33380
25982025-01-10T22:05:06.126ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33380 req_id = 952e3c64-6ef8-4edc-8509-172cab3ae268 response_code = 200 uri = /vpc-routes
25992025-01-10T22:05:06.141ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
26002025-01-10T22:05:06.141ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 72 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:33380 req_id = a320a02c-1af2-4546-8583-5ac56cda709f response_code = 204 uri = /eip-gateways
26012025-01-10T22:05:06.149ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 207 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:33380 req_id = 2792b5d9-c12b-48a8-89b8-faa2d7e1cc5e response_code = 204 uri = /vpc-routes
26022025-01-10T22:05:06.215ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58978
26032025-01-10T22:05:06.216ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58978 req_id = 2b8c0589-01e5-4c11-9ea5-237c92dceed8 response_code = 200 uri = /v2p
26042025-01-10T22:05:06.237ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53418
26052025-01-10T22:05:06.237ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
26062025-01-10T22:05:06.278ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
26072025-01-10T22:05:06.334ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97039 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:53418 req_id = 4c3ae2c4-8afe-40cd-a119-7397871756d7 response_code = 204 uri = /switch-ports
26082025-01-10T22:05:06.335ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53418 req_id = 643744f5-b520-403f-b562-bc6f78c022a7 response_code = 200 uri = /network-bootstore-config
26092025-01-10T22:05:17.688ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7594 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = eba8f191-9b30-4d28-8102-6620652cd4da response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
26102025-01-10T22:05:25.357ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:39255
26112025-01-10T22:05:25.357ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:39255 req_id = 934f32aa-00f0-489c-851e-401714e3339e response_code = 200 uri = /baseboard
26122025-01-10T22:05:35.176ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44634
26132025-01-10T22:05:35.176ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44634 req_id = 2bce341b-53b5-4c11-8fca-499467be8f52 response_code = 200 uri = /vpc-routes
26142025-01-10T22:05:35.191ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
26152025-01-10T22:05:35.191ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44634 req_id = 6148e0de-57bf-4576-999d-fd49505c2c1f response_code = 204 uri = /eip-gateways
26162025-01-10T22:05:35.199ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 206 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44634 req_id = f63a8e19-a824-4596-823e-5634990cb042 response_code = 204 uri = /vpc-routes
26172025-01-10T22:05:35.307ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64018
26182025-01-10T22:05:35.307ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 120 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64018 req_id = ac191e51-316f-407e-a574-2be8e9de51a4 response_code = 200 uri = /v2p
26192025-01-10T22:05:35.361ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51273
26202025-01-10T22:05:35.361ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
26212025-01-10T22:05:35.400ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
26222025-01-10T22:05:35.448ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87127 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:51273 req_id = 20985806-52b6-4100-a625-3b1e92c36246 response_code = 204 uri = /switch-ports
26232025-01-10T22:05:35.452ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51273 req_id = 68e26482-0733-4312-a560-9ed4f5632acd response_code = 200 uri = /network-bootstore-config
26242025-01-10T22:05:35.954ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34046
26252025-01-10T22:05:35.954ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34046 req_id = f6ceaa42-5904-46b2-b189-8c5372db1bcc response_code = 200 uri = /vpc-routes
26262025-01-10T22:05:35.975ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
26272025-01-10T22:05:35.975ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34046 req_id = 739702a0-9b49-4cc8-b4f2-5d41a7a5714c response_code = 204 uri = /eip-gateways
26282025-01-10T22:05:35.991ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 220 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34046 req_id = 250e3eb5-3056-4887-8980-3e32a84527a0 response_code = 204 uri = /vpc-routes
26292025-01-10T22:05:36.115ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55891
26302025-01-10T22:05:36.116ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 160 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55891 req_id = 6147445a-536b-4f45-81f9-a7f17508eca1 response_code = 200 uri = /v2p
26312025-01-10T22:05:36.120ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41124
26322025-01-10T22:05:36.120ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41124 req_id = 36bbc813-c643-4a50-beee-827ed60834b2 response_code = 200 uri = /vpc-routes
26332025-01-10T22:05:36.189ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
26342025-01-10T22:05:36.189ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:41124 req_id = 4205523e-b9d1-4295-8b2e-1a90a7c6b4e0 response_code = 204 uri = /eip-gateways
26352025-01-10T22:05:36.199ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 240 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:41124 req_id = 94ba6cc8-12d3-4fcc-a4ce-d8a34e43cabf response_code = 204 uri = /vpc-routes
26362025-01-10T22:05:36.288ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46485
26372025-01-10T22:05:36.288ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
26382025-01-10T22:05:36.325ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
26392025-01-10T22:05:36.379ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91238 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:46485 req_id = b97e8b41-d70c-4c1a-83c1-f045a470deab response_code = 204 uri = /switch-ports
26402025-01-10T22:05:36.379ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53281
26412025-01-10T22:05:36.380ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 183 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53281 req_id = 95b4c52e-2bda-4f10-b9c9-3d401d511909 response_code = 200 uri = /v2p
26422025-01-10T22:05:36.380ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 220 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46485 req_id = d824aaaa-0beb-430e-812a-4249c313055e response_code = 200 uri = /network-bootstore-config
26432025-01-10T22:05:36.421ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60404
26442025-01-10T22:05:36.421ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
26452025-01-10T22:05:36.455ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
26462025-01-10T22:05:36.511ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90589 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:60404 req_id = ebabd6b6-9ff3-472f-a861-8f364458a2af response_code = 204 uri = /switch-ports
26472025-01-10T22:05:36.512ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60404 req_id = a46eac80-5793-4bfa-827a-5f5c30b44ed1 response_code = 200 uri = /network-bootstore-config
26482025-01-10T22:05:47.688ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7446 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = 3f5fde4d-9d4c-4a07-9a79-19684977ec0b response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
26492025-01-10T22:05:55.389ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:36089
26502025-01-10T22:05:55.389ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 74 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:36089 req_id = bf44304c-4847-4195-aedf-1757fcb3cd14 response_code = 200 uri = /baseboard
26512025-01-10T22:06:05.042ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59924
26522025-01-10T22:06:05.042ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 52 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59924 req_id = c6fda4e6-edd8-4fcd-a769-f68f78830651 response_code = 200 uri = /vpc-routes
26532025-01-10T22:06:05.065ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
26542025-01-10T22:06:05.065ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:59924 req_id = c0b1283e-48ab-41df-8a41-6456786dfdaf response_code = 204 uri = /eip-gateways
26552025-01-10T22:06:05.073ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 203 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:59924 req_id = b616aea7-afb9-4ff1-9642-9279bfbebd1b response_code = 204 uri = /vpc-routes
26562025-01-10T22:06:05.240ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61776
26572025-01-10T22:06:05.240ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61776 req_id = 292dfde8-2758-4102-864f-1f45b7f89d96 response_code = 200 uri = /v2p
26582025-01-10T22:06:05.257ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47340
26592025-01-10T22:06:05.257ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
26602025-01-10T22:06:05.295ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
26612025-01-10T22:06:05.349ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91738 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:47340 req_id = 3cc58429-9554-4650-be09-8898758c8158 response_code = 204 uri = /switch-ports
26622025-01-10T22:06:05.349ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 209 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47340 req_id = 667954cf-baac-41b2-b516-4a711b4ffdd6 response_code = 200 uri = /network-bootstore-config
26632025-01-10T22:06:05.964ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45743
26642025-01-10T22:06:05.964ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45743 req_id = 8c053a40-c5cb-44bf-990e-c4a20a308b37 response_code = 200 uri = /vpc-routes
26652025-01-10T22:06:06.016ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
26662025-01-10T22:06:06.016ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45743 req_id = e85671ca-5dcc-407e-bfaa-3d926ac5e4ea response_code = 204 uri = /eip-gateways
26672025-01-10T22:06:06.036ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 205 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45743 req_id = 427e0a7f-4d9f-4e04-99aa-fa37bda84085 response_code = 204 uri = /vpc-routes
26682025-01-10T22:06:06.155ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52255
26692025-01-10T22:06:06.155ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52255 req_id = 262d2862-0be4-4f5d-92ac-fadb45431b9f response_code = 200 uri = /vpc-routes
26702025-01-10T22:06:06.186ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50818
26712025-01-10T22:06:06.186ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 146 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50818 req_id = 0de2c9d9-fd7b-449d-a3b6-0a5dd45e01e5 response_code = 200 uri = /v2p
26722025-01-10T22:06:06.192ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54026
26732025-01-10T22:06:06.192ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
26742025-01-10T22:06:06.231ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
26752025-01-10T22:06:06.286ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93808 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:54026 req_id = 2bdbe29b-4d4d-4aab-9b5a-f373eceffca8 response_code = 204 uri = /switch-ports
26762025-01-10T22:06:06.286ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55757
26772025-01-10T22:06:06.286ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
26782025-01-10T22:06:06.286ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 175 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52255 req_id = f2e278ee-c676-4ef9-9e80-d0cb0836165a response_code = 204 uri = /eip-gateways
26792025-01-10T22:06:06.286ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 220 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55757 req_id = 9e8170d9-2f12-46a8-aeb7-5721d42ac024 response_code = 200 uri = /v2p
26802025-01-10T22:06:06.286ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 238 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54026 req_id = 1ca9db65-9180-40db-8fa7-db6f39347680 response_code = 200 uri = /network-bootstore-config
26812025-01-10T22:06:06.299ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 282 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52255 req_id = 05d25abe-e4b7-4c09-9070-8771c33842e8 response_code = 204 uri = /vpc-routes
26822025-01-10T22:06:06.304ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54743
26832025-01-10T22:06:06.304ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
26842025-01-10T22:06:06.342ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
26852025-01-10T22:06:06.397ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92814 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:54743 req_id = 69234413-6a99-4dac-b4da-6793f4fba946 response_code = 204 uri = /switch-ports
26862025-01-10T22:06:06.398ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 171 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54743 req_id = 2966e777-6aef-4b42-84e7-bc5f65ecde5c response_code = 200 uri = /network-bootstore-config
26872025-01-10T22:06:17.688ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7485 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = f387bf83-99b0-4cee-9a53-25acc5889529 response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
26882025-01-10T22:06:25.421ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:50463
26892025-01-10T22:06:25.421ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:50463 req_id = dcf4429d-dd05-4be8-9c5d-2281cd2bc1e6 response_code = 200 uri = /baseboard
26902025-01-10T22:06:35.055ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61134
26912025-01-10T22:06:35.055ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61134 req_id = 49c71fb3-9f48-403b-b0b7-f7a607d96acf response_code = 200 uri = /vpc-routes
26922025-01-10T22:06:35.114ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
26932025-01-10T22:06:35.114ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61134 req_id = d5777635-2f68-4d13-b8a3-992e5f183b88 response_code = 204 uri = /eip-gateways
26942025-01-10T22:06:35.122ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 239 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61134 req_id = a778ec02-b5dd-4e55-99d8-5aa0aceb3b23 response_code = 204 uri = /vpc-routes
26952025-01-10T22:06:35.302ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42429
26962025-01-10T22:06:35.302ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
26972025-01-10T22:06:35.302ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
26982025-01-10T22:06:35.302ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-zones.json file = common/src/ledger.rs:177
26992025-01-10T22:06:35.303ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-zones.json file = common/src/ledger.rs:177
27002025-01-10T22:06:35.315ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27012025-01-10T22:06:35.315ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_73828a50-9586-442b-b526-668d3e1de808
27022025-01-10T22:06:35.340ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27032025-01-10T22:06:35.340ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_598e3879-d23d-4354-9e0c-045503f010a6
27042025-01-10T22:06:35.364ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27052025-01-10T22:06:35.364ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_5d7ad0a3-e47e-4369-b599-d02dbd3a8829
27062025-01-10T22:06:35.384ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27072025-01-10T22:06:35.384ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_d27ec511-4e48-4e26-9677-295de0452e91
27082025-01-10T22:06:35.405ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59011
27092025-01-10T22:06:35.405ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 189 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59011 req_id = 5395e4cb-556d-42a4-9e72-7e503fe17907 response_code = 200 uri = /v2p
27102025-01-10T22:06:35.406ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27112025-01-10T22:06:35.406ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_ce5c629d-252d-4e12-9f9d-2087cb772d81
27122025-01-10T22:06:35.427ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27132025-01-10T22:06:35.427ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_7b0e1b76-af4f-49bd-9848-611d8d8666ad
27142025-01-10T22:06:35.450ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27152025-01-10T22:06:35.450ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_619e5037-b196-43f6-b092-666b1b1387ff
27162025-01-10T22:06:35.472ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27172025-01-10T22:06:35.472ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_eecab578-d379-46e4-90f5-9dc11839dad5
27182025-01-10T22:06:35.480ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43331
27192025-01-10T22:06:35.480ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
27202025-01-10T22:06:35.498ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27212025-01-10T22:06:35.498ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_784c4a6b-3804-44fa-afd4-c974bef9dd5d
27222025-01-10T22:06:35.514ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 211914 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42429 req_id = 4feaeee9-0450-4c9a-a17b-b19e71e69110 response_code = 200 uri = /inventory
27232025-01-10T22:06:35.518ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
27242025-01-10T22:06:35.549ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41630
27252025-01-10T22:06:35.549ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6decc476-dbff-4cb1-a909-d9d4408adaab, kind: Service { id: f44690fa-03a0-42b0-bb97-26c3d20aac30 }, name: Name("external-dns-f44690fa-03a0-42b0-bb97-26c3d20aac30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 143])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6f22ade2-da2d-4cbc-9141-adcc3b335aed, kind: Service { id: bce93894-738c-4596-b983-281da931e977 }, name: Name("external-dns-bce93894-738c-4596-b983-281da931e977"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 88])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 8448bae1-35b5-443a-8e61-6f2848603054, kind: Service { id: 4106fd29-fc2d-4538-b321-136cbc8ea9d2 }, name: Name("nexus-4106fd29-fc2d-4538-b321-136cbc8ea9d2"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 27])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9676df2d-b2c5-40f4-ae20-30ad4de6c8e2, kind: Service { id: 19558293-ba4c-462e-8746-8d928c3a9612 }, name: Name("nexus-19558293-ba4c-462e-8746-8d928c3a9612"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 42])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bf22f522-6412-4896-90b5-9bc8aab0192d, kind: Service { id: b507bc15-7e7b-4d56-a806-1da438b76052 }, name: Name("nexus-b507bc15-7e7b-4d56-a806-1da438b76052"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 137])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
27262025-01-10T22:06:35.549ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27272025-01-10T22:06:35.549ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27282025-01-10T22:06:35.549ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27292025-01-10T22:06:35.549ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27302025-01-10T22:06:35.549ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
27312025-01-10T22:06:35.549ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27322025-01-10T22:06:35.549ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 430 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41630 req_id = 1122c758-79c5-4609-b879-52c40d70341a response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
27332025-01-10T22:06:35.575ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95067 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:43331 req_id = f26b8e9a-c5eb-4acc-9331-fb1438f506ba response_code = 204 uri = /switch-ports
27342025-01-10T22:06:35.576ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43331 req_id = 9d6a0ff0-9c04-4077-ad3e-cae95837378c response_code = 200 uri = /network-bootstore-config
27352025-01-10T22:06:36.252ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34137
27362025-01-10T22:06:36.252ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34137 req_id = 28d1b2ec-e3ca-496b-afb8-c0a4d2d592de response_code = 200 uri = /vpc-routes
27372025-01-10T22:06:36.323ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61135
27382025-01-10T22:06:36.323ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61135 req_id = 00e937f4-840d-4681-a094-d947a5ebdff4 response_code = 200 uri = /vpc-routes
27392025-01-10T22:06:36.334ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56055
27402025-01-10T22:06:36.334ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56055 req_id = d3c59c20-7870-4f5d-b64f-c89b1c7ec67d response_code = 200 uri = /v2p
27412025-01-10T22:06:36.377ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
27422025-01-10T22:06:36.377ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34137 req_id = e09c416c-4bc7-4d9f-a106-82d520978b0d response_code = 204 uri = /eip-gateways
27432025-01-10T22:06:36.438ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41152
27442025-01-10T22:06:36.438ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6decc476-dbff-4cb1-a909-d9d4408adaab, kind: Service { id: f44690fa-03a0-42b0-bb97-26c3d20aac30 }, name: Name("external-dns-f44690fa-03a0-42b0-bb97-26c3d20aac30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 143])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6f22ade2-da2d-4cbc-9141-adcc3b335aed, kind: Service { id: bce93894-738c-4596-b983-281da931e977 }, name: Name("external-dns-bce93894-738c-4596-b983-281da931e977"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 88])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 8448bae1-35b5-443a-8e61-6f2848603054, kind: Service { id: 4106fd29-fc2d-4538-b321-136cbc8ea9d2 }, name: Name("nexus-4106fd29-fc2d-4538-b321-136cbc8ea9d2"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 27])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9676df2d-b2c5-40f4-ae20-30ad4de6c8e2, kind: Service { id: 19558293-ba4c-462e-8746-8d928c3a9612 }, name: Name("nexus-19558293-ba4c-462e-8746-8d928c3a9612"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 42])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bf22f522-6412-4896-90b5-9bc8aab0192d, kind: Service { id: b507bc15-7e7b-4d56-a806-1da438b76052 }, name: Name("nexus-b507bc15-7e7b-4d56-a806-1da438b76052"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 137])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
27452025-01-10T22:06:36.439ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27462025-01-10T22:06:36.439ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27472025-01-10T22:06:36.439ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27482025-01-10T22:06:36.439ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27492025-01-10T22:06:36.439ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
27502025-01-10T22:06:36.439ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27512025-01-10T22:06:36.439ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 335 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:41152 req_id = f11271a6-2ee1-4541-8a73-8f7321c1aff2 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
27522025-01-10T22:06:36.448ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 226 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34137 req_id = a78dfa66-3b10-4174-9fad-e35eb0c88d77 response_code = 204 uri = /vpc-routes
27532025-01-10T22:06:36.537ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58576
27542025-01-10T22:06:36.537ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
27552025-01-10T22:06:36.537ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
27562025-01-10T22:06:36.537ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-zones.json file = common/src/ledger.rs:177
27572025-01-10T22:06:36.537ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-zones.json file = common/src/ledger.rs:177
27582025-01-10T22:06:36.549ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27592025-01-10T22:06:36.549ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_73828a50-9586-442b-b526-668d3e1de808
27602025-01-10T22:06:36.552ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61317
27612025-01-10T22:06:36.553ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
27622025-01-10T22:06:36.574ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27632025-01-10T22:06:36.574ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_598e3879-d23d-4354-9e0c-045503f010a6
27642025-01-10T22:06:36.588ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51537
27652025-01-10T22:06:36.592ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
27662025-01-10T22:06:36.592ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 670 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61135 req_id = 5e40d88a-7241-40f7-80b5-3d890034e483 response_code = 204 uri = /eip-gateways
27672025-01-10T22:06:36.592ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 257 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51537 req_id = a159b3bc-63d1-4dd8-bb9e-11e1db63945a response_code = 200 uri = /v2p
27682025-01-10T22:06:36.593ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
27692025-01-10T22:06:36.601ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 359 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61135 req_id = b78776bd-b524-4bd3-9094-6e354a2d82db response_code = 204 uri = /vpc-routes
27702025-01-10T22:06:36.603ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27712025-01-10T22:06:36.603ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_5d7ad0a3-e47e-4369-b599-d02dbd3a8829
27722025-01-10T22:06:36.627ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27732025-01-10T22:06:36.627ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_d27ec511-4e48-4e26-9677-295de0452e91
27742025-01-10T22:06:36.651ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27752025-01-10T22:06:36.651ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_ce5c629d-252d-4e12-9f9d-2087cb772d81
27762025-01-10T22:06:36.655ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102529 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:61317 req_id = d095ace8-1a38-420e-8e81-e45d5b178a50 response_code = 204 uri = /switch-ports
27772025-01-10T22:06:36.656ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 262 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61317 req_id = 223443f1-94ba-46c5-a3ad-ad5824791e65 response_code = 200 uri = /network-bootstore-config
27782025-01-10T22:06:36.675ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27792025-01-10T22:06:36.675ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_7b0e1b76-af4f-49bd-9848-611d8d8666ad
27802025-01-10T22:06:36.695ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27812025-01-10T22:06:36.695ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_619e5037-b196-43f6-b092-666b1b1387ff
27822025-01-10T22:06:36.715ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27832025-01-10T22:06:36.715ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_eecab578-d379-46e4-90f5-9dc11839dad5
27842025-01-10T22:06:36.734ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27852025-01-10T22:06:36.734ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_784c4a6b-3804-44fa-afd4-c974bef9dd5d
27862025-01-10T22:06:36.742ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41902
27872025-01-10T22:06:36.745ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
27882025-01-10T22:06:36.745ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
27892025-01-10T22:06:36.745ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/config/omicron-zones.json file = common/src/ledger.rs:177
27902025-01-10T22:06:36.746ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/10481327-13d1-4548-950f-130d063e2690/config/omicron-zones.json file = common/src/ledger.rs:177
27912025-01-10T22:06:36.746ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 209776 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58576 req_id = 77767679-7050-4f01-8de2-890fa938482f response_code = 200 uri = /inventory
27922025-01-10T22:06:36.755ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44233
27932025-01-10T22:06:36.755ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
27942025-01-10T22:06:36.759ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 73828a50-9586-442b-b526-668d3e1de808 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27952025-01-10T22:06:36.759ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_73828a50-9586-442b-b526-668d3e1de808
27962025-01-10T22:06:36.789ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 598e3879-d23d-4354-9e0c-045503f010a6 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27972025-01-10T22:06:36.789ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_598e3879-d23d-4354-9e0c-045503f010a6
27982025-01-10T22:06:36.794ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
27992025-01-10T22:06:36.817ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 5d7ad0a3-e47e-4369-b599-d02dbd3a8829 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28002025-01-10T22:06:36.817ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_5d7ad0a3-e47e-4369-b599-d02dbd3a8829
28012025-01-10T22:06:36.839ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43429
28022025-01-10T22:06:36.839ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: d27ec511-4e48-4e26-9677-295de0452e91 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28032025-01-10T22:06:36.839ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_d27ec511-4e48-4e26-9677-295de0452e91
28042025-01-10T22:06:36.843ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 6decc476-dbff-4cb1-a909-d9d4408adaab, kind: Service { id: f44690fa-03a0-42b0-bb97-26c3d20aac30 }, name: Name("external-dns-f44690fa-03a0-42b0-bb97-26c3d20aac30"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 175, 143])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 6f22ade2-da2d-4cbc-9141-adcc3b335aed, kind: Service { id: bce93894-738c-4596-b983-281da931e977 }, name: Name("external-dns-bce93894-738c-4596-b983-281da931e977"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 88])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 8448bae1-35b5-443a-8e61-6f2848603054, kind: Service { id: 4106fd29-fc2d-4538-b321-136cbc8ea9d2 }, name: Name("nexus-4106fd29-fc2d-4538-b321-136cbc8ea9d2"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 221, 27])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9676df2d-b2c5-40f4-ae20-30ad4de6c8e2, kind: Service { id: 19558293-ba4c-462e-8746-8d928c3a9612 }, name: Name("nexus-19558293-ba4c-462e-8746-8d928c3a9612"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 160, 42])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: bf22f522-6412-4896-90b5-9bc8aab0192d, kind: Service { id: b507bc15-7e7b-4d56-a806-1da438b76052 }, name: Name("nexus-b507bc15-7e7b-4d56-a806-1da438b76052"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 241, 137])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
28052025-01-10T22:06:36.843ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
28062025-01-10T22:06:36.843ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
28072025-01-10T22:06:36.843ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
28082025-01-10T22:06:36.843ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
28092025-01-10T22:06:36.843ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
28102025-01-10T22:06:36.843ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
28112025-01-10T22:06:36.843ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4434 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43429 req_id = d8226246-92ff-4ae2-b62b-b3595323a097 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
28122025-01-10T22:06:36.860ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105726 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:44233 req_id = c795544f-14b7-4f4a-8fca-6d12c10ac453 response_code = 204 uri = /switch-ports
28132025-01-10T22:06:36.861ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 156 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44233 req_id = 5a3525e5-5efc-43bb-be46-73f5b6cc9934 response_code = 200 uri = /network-bootstore-config
28142025-01-10T22:06:36.865ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: ce5c629d-252d-4e12-9f9d-2087cb772d81 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28152025-01-10T22:06:36.865ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_ce5c629d-252d-4e12-9f9d-2087cb772d81
28162025-01-10T22:06:36.888ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 7b0e1b76-af4f-49bd-9848-611d8d8666ad (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28172025-01-10T22:06:36.888ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_7b0e1b76-af4f-49bd-9848-611d8d8666ad
28182025-01-10T22:06:36.913ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 619e5037-b196-43f6-b092-666b1b1387ff (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28192025-01-10T22:06:36.913ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_619e5037-b196-43f6-b092-666b1b1387ff
28202025-01-10T22:06:36.934ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: eecab578-d379-46e4-90f5-9dc11839dad5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28212025-01-10T22:06:36.934ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_eecab578-d379-46e4-90f5-9dc11839dad5
28222025-01-10T22:06:36.954ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 784c4a6b-3804-44fa-afd4-c974bef9dd5d (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28232025-01-10T22:06:36.954ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_784c4a6b-3804-44fa-afd4-c974bef9dd5d
28242025-01-10T22:06:36.968ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 225507 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41902 req_id = 8daa1555-ca21-4afd-86bc-bc48a3418d05 response_code = 200 uri = /inventory
28252025-01-10T22:06:47.688ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7230 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = 8c18d509-d1c0-42d6-bb71-4c0140aa851e response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
28262025-01-10T22:06:55.452ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:37903
28272025-01-10T22:06:55.452ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:37903 req_id = f0f71653-3b28-474a-8006-dcbadd67efb3 response_code = 200 uri = /baseboard
28282025-01-10T22:07:05.081ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35084
28292025-01-10T22:07:05.082ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35084 req_id = 74446eea-c232-4a94-9745-87dde42b3b0e response_code = 200 uri = /vpc-routes
28302025-01-10T22:07:05.121ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
28312025-01-10T22:07:05.121ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35084 req_id = ce5d13a6-3ca7-4eee-8604-ebfb13a3a92a response_code = 204 uri = /eip-gateways
28322025-01-10T22:07:05.130ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 168 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35084 req_id = 8b0b76d2-6435-4340-b139-50c8c52447b3 response_code = 204 uri = /vpc-routes
28332025-01-10T22:07:05.154ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52712
28342025-01-10T22:07:05.154ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52712 req_id = 340eb7f4-d4b6-4ea8-932d-9d6548043ab4 response_code = 200 uri = /v2p
28352025-01-10T22:07:05.169ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48368
28362025-01-10T22:07:05.169ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
28372025-01-10T22:07:05.207ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
28382025-01-10T22:07:05.263ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94181 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:48368 req_id = ca34a5b8-a6c1-418f-9527-be25b8176cc7 response_code = 204 uri = /switch-ports
28392025-01-10T22:07:05.263ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48368 req_id = e7574676-ae95-48db-b620-d118f6e7e451 response_code = 200 uri = /network-bootstore-config
28402025-01-10T22:07:05.934ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39047
28412025-01-10T22:07:05.934ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39047 req_id = 4de0d257-f857-4c2a-8261-b6c4e6cd9510 response_code = 200 uri = /vpc-routes
28422025-01-10T22:07:05.986ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
28432025-01-10T22:07:05.987ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39047 req_id = e76bf134-83fc-4927-b1db-4764a011251e response_code = 204 uri = /eip-gateways
28442025-01-10T22:07:06.015ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55133
28452025-01-10T22:07:06.015ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 161 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55133 req_id = 48d03e97-76cd-464b-8823-ab2546a0e98c response_code = 200 uri = /v2p
28462025-01-10T22:07:06.025ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 299 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39047 req_id = e76f5ecf-fd4c-4188-8a19-0082fc099edb response_code = 204 uri = /vpc-routes
28472025-01-10T22:07:06.101ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65373
28482025-01-10T22:07:06.101ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
28492025-01-10T22:07:06.138ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
28502025-01-10T22:07:06.191ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90141 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:65373 req_id = 4cb8e2ab-423e-43b8-8436-4b760eace2f9 response_code = 204 uri = /switch-ports
28512025-01-10T22:07:06.191ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41256
28522025-01-10T22:07:06.192ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41256 req_id = 79143603-ae16-40a0-84d7-968ca0802d78 response_code = 200 uri = /vpc-routes
28532025-01-10T22:07:06.192ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 198 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65373 req_id = da57e4b7-5a13-4bc9-ac27-b8a26cda04be response_code = 200 uri = /network-bootstore-config
28542025-01-10T22:07:06.232ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
28552025-01-10T22:07:06.232ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 71 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:41256 req_id = e5271fe0-843d-4f81-80b9-6e0d4bf27201 response_code = 204 uri = /eip-gateways
28562025-01-10T22:07:06.240ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64326
28572025-01-10T22:07:06.240ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64326 req_id = 2012add5-c504-4484-93ea-687cae285811 response_code = 200 uri = /v2p
28582025-01-10T22:07:06.248ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 319 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:41256 req_id = 46fb93ea-ba28-4521-a60c-2897a8333ec1 response_code = 204 uri = /vpc-routes
28592025-01-10T22:07:06.284ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64015
28602025-01-10T22:07:06.285ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
28612025-01-10T22:07:06.324ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
28622025-01-10T22:07:06.378ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93314 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:64015 req_id = 8cb11291-25f2-4c74-8674-82be6f2531cb response_code = 204 uri = /switch-ports
28632025-01-10T22:07:06.378ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64015 req_id = c7a052fd-7b81-4e62-a913-7f862e495fc3 response_code = 200 uri = /network-bootstore-config
28642025-01-10T22:07:17.687ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7034 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = 2065f260-50d5-4931-9e84-5a95069423d0 response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
28652025-01-10T22:07:25.486ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:52218
28662025-01-10T22:07:25.486ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 75 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:52218 req_id = 3cafcdd3-96d1-4ab7-b49d-4bc9326abdae response_code = 200 uri = /baseboard
28672025-01-10T22:07:35.155ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36791
28682025-01-10T22:07:35.155ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36791 req_id = 998afbbb-3631-4e15-a273-cc1d28d9ad7f response_code = 200 uri = /vpc-routes
28692025-01-10T22:07:35.199ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
28702025-01-10T22:07:35.199ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 81 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:36791 req_id = e09cb108-29ee-4583-9b0e-623fadc93290 response_code = 204 uri = /eip-gateways
28712025-01-10T22:07:35.226ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63629
28722025-01-10T22:07:35.226ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63629 req_id = 00af0b24-a3cb-475d-9668-6ab438138999 response_code = 200 uri = /v2p
28732025-01-10T22:07:35.229ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:36791 req_id = bf0582b4-d4dc-4792-8c52-17dec2bb01e6 response_code = 204 uri = /vpc-routes
28742025-01-10T22:07:35.505ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59637
28752025-01-10T22:07:35.505ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
28762025-01-10T22:07:35.540ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
28772025-01-10T22:07:35.595ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90193 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:59637 req_id = 7e4e885d-7875-4bd9-8db3-50bd96f36788 response_code = 204 uri = /switch-ports
28782025-01-10T22:07:35.599ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59637 req_id = 4e25597b-2c84-42e6-ae95-76c4139c6cc9 response_code = 200 uri = /network-bootstore-config
28792025-01-10T22:07:36.070ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35761
28802025-01-10T22:07:36.070ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 54 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35761 req_id = 03497aa2-7f1f-4a21-86d1-9019c8fd1428 response_code = 200 uri = /vpc-routes
28812025-01-10T22:07:36.097ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
28822025-01-10T22:07:36.097ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35761 req_id = 84ea5f7c-13b2-4be4-b8fd-45e21e1cd386 response_code = 204 uri = /eip-gateways
28832025-01-10T22:07:36.108ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 301 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35761 req_id = 21ff743a-b373-4852-9705-7b460c817f26 response_code = 204 uri = /vpc-routes
28842025-01-10T22:07:36.171ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36435
28852025-01-10T22:07:36.171ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36435 req_id = 6e72146f-2d8c-4fe3-9974-cb4965b1699f response_code = 200 uri = /v2p
28862025-01-10T22:07:36.216ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59659
28872025-01-10T22:07:36.216ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
28882025-01-10T22:07:36.251ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
28892025-01-10T22:07:36.302ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 85784 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59659 req_id = 0d982cef-4e7f-4652-ab39-7da6539af0d3 response_code = 204 uri = /switch-ports
28902025-01-10T22:07:36.302ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57938
28912025-01-10T22:07:36.302ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57938 req_id = d6a392d9-7c2e-4de4-b578-2a633d7dcd20 response_code = 200 uri = /vpc-routes
28922025-01-10T22:07:36.302ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 180 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59659 req_id = b6a6472d-a992-4029-a18b-779b3aec1de9 response_code = 200 uri = /network-bootstore-config
28932025-01-10T22:07:36.320ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
28942025-01-10T22:07:36.320ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57938 req_id = c59a8a7e-3651-423c-803f-3f90acbe6a7d response_code = 204 uri = /eip-gateways
28952025-01-10T22:07:36.329ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 341 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57938 req_id = 1d577cb4-7478-46ff-b424-4f3060f8be71 response_code = 204 uri = /vpc-routes
28962025-01-10T22:07:36.387ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34044
28972025-01-10T22:07:36.387ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34044 req_id = ef8ba277-2b91-49c8-84d5-eaf555138c0c response_code = 200 uri = /v2p
28982025-01-10T22:07:36.451ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54963
28992025-01-10T22:07:36.451ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
29002025-01-10T22:07:36.488ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
29012025-01-10T22:07:36.541ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89658 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:54963 req_id = 243b3277-ce73-4969-888f-da4e84cb98d8 response_code = 204 uri = /switch-ports
29022025-01-10T22:07:36.541ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54963 req_id = 79d5ab60-8792-4d7f-ba8e-e15c51ffd7d5 response_code = 200 uri = /network-bootstore-config
29032025-01-10T22:07:47.687ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7007 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = a9616238-6b7b-44b6-9b51-1e4ead56fe86 response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
29042025-01-10T22:07:55.520ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:63216
29052025-01-10T22:07:55.520ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 77 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:63216 req_id = 8ddddc18-0b08-4b7d-9c83-163bcca35637 response_code = 200 uri = /baseboard
29062025-01-10T22:07:57.518ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54157
29072025-01-10T22:07:57.519ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 179 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54157 req_id = eb0c30f6-9578-4f03-af66-87ba10924591 response_code = 200 uri = /v2p
29082025-01-10T22:07:57.520ZINFOSledAgent (PortManager): Mapping virtual NIC to physical host file = illumos-utils/src/opte/port_manager.rs:874 mapping = VirtualNetworkInterfaceHost { virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), physical_host_ip: fd00:1122:3344:101::1, vni: Vni(14455117) }
29092025-01-10T22:07:57.521ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2212 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:54157 req_id = 84c85286-1717-44d7-b73f-93d76d4fba66 response_code = 204 uri = /v2p
29102025-01-10T22:07:57.613ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35697
29112025-01-10T22:07:57.618ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:555 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: dad6653d-2a0a-4a4a-bb5e-32d25108dad3, kind: Instance { id: 191fa903-5a42-4777-b82a-12d4334e9b89 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(14455117), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: dad6653d-2a0a-4a4a-bb5e-32d25108dad3, kind: Instance { id: 191fa903-5a42-4777-b82a-12d4334e9b89 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(14455117), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: dad6653d-2a0a-4a4a-bb5e-32d25108dad3, kind: Instance { id: 191fa903-5a42-4777-b82a-12d4334e9b89 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(14455117), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(14455117))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: dad6653d-2a0a-4a4a-bb5e-32d25108dad3, kind: Instance { id: 191fa903-5a42-4777-b82a-12d4334e9b89 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(14455117), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [InstanceDisk { disk_id: c9893bd8-7a91-411c-be2e-a31af1914823, slot: 0, read_only: false, vcr_json: .., name: "disk-929a6cd84e4" }], boot_settings: Some(InstanceBootSettings { order: [c9893bd8-7a91-411c-be2e-a31af1914823] }), cloud_init_bytes: Some(..) } instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 metadata = InstanceMetadata { silo_id: 14fadc0e-36c7-4d02-903f-a5694aef3aa6, project_id: b598fc04-690b-4490-ba1b-68ac14745390 } migration_id = None propolis_addr = [fd00:1122:3344:101::1:0]:12400 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38 vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2025-01-10T22:07:57.611209959Z }
29122025-01-10T22:07:57.618ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:588 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 migration_id = None propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29132025-01-10T22:07:57.618ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:1562 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 migration_id = None propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38 state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: dad6653d-2a0a-4a4a-bb5e-32d25108dad3, kind: Instance { id: 191fa903-5a42-4777-b82a-12d4334e9b89 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(14455117), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: dad6653d-2a0a-4a4a-bb5e-32d25108dad3, kind: Instance { id: 191fa903-5a42-4777-b82a-12d4334e9b89 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(14455117), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: dad6653d-2a0a-4a4a-bb5e-32d25108dad3, kind: Instance { id: 191fa903-5a42-4777-b82a-12d4334e9b89 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(14455117), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(14455117))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: dad6653d-2a0a-4a4a-bb5e-32d25108dad3, kind: Instance { id: 191fa903-5a42-4777-b82a-12d4334e9b89 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(14455117), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [InstanceDisk { disk_id: c9893bd8-7a91-411c-be2e-a31af1914823, slot: 0, read_only: false, vcr_json: .., name: "disk-929a6cd84e4" }], boot_settings: Some(InstanceBootSettings { order: [c9893bd8-7a91-411c-be2e-a31af1914823] }), cloud_init_bytes: Some(..) }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2025-01-10T22:07:57.611209959Z }, propolis_addr: [fd00:1122:3344:101::1:0]:12400, migration_id: None }
29142025-01-10T22:07:57.621ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7364 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35697 req_id = 2cd09365-0db4-42b0-9925-06c30717a961 response_code = 200 uri = /vmms/1aed9130-b669-4f4f-972c-06e21de1fc38
29152025-01-10T22:07:57.697ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52321
29162025-01-10T22:07:57.697ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 469 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52321 req_id = 1f72cd23-74bb-48e0-8ecf-6b2b6b84771e response_code = 200 uri = /vmms/1aed9130-b669-4f4f-972c-06e21de1fc38/state
29172025-01-10T22:07:57.699ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte6", ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 240, 0, 0]), slot: 0, vni: Vni { inner: 14455117 }, subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 } }) }
29182025-01-10T22:07:57.699ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
29192025-01-10T22:07:57.734ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38 file = illumos-utils/src/zone.rs:325 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29202025-01-10T22:07:57.755ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38 file = illumos-utils/src/zone.rs:363 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29212025-01-10T22:07:57.801ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53958
29222025-01-10T22:07:57.801ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53958 req_id = 419070dc-9cf9-41a1-81cf-c597262ddffe response_code = 200 uri = /vpc-routes
29232025-01-10T22:07:57.816ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {dad6653d-2a0a-4a4a-bb5e-32d25108dad3: {10.151.2.175: {29ac68e7-94b9-4881-adaa-290583c8a55e}, 10.151.2.176: {29ac68e7-94b9-4881-adaa-290583c8a55e}}} }
29242025-01-10T22:07:59.040ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlInstance0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1:0'/> <propval type="astring" name="listen_port" value='12400'/> <propval type="astring" name="metric_addr" value='dns'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29252025-01-10T22:07:59.040ZINFOSledAgent (InstanceManager): Booting oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38 zone file = illumos-utils/src/running_zone.rs:488 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38 zone = oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38
29262025-01-10T22:07:59.284ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38") failed: Property not found. retry in 35.837342ms file = illumos-utils/src/svc.rs:36 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38 zone = oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38
29272025-01-10T22:07:59.329ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38") failed: Property not found. retry in 61.873432ms file = illumos-utils/src/svc.rs:36 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38 zone = oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38
29282025-01-10T22:07:59.407ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38") failed: Property not found. retry in 264.790944ms file = illumos-utils/src/svc.rs:36 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38 zone = oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38
29292025-01-10T22:07:59.684ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38") failed: Property not found. retry in 594.618941ms file = illumos-utils/src/svc.rs:36 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38 zone = oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38
29302025-01-10T22:08:00.294ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38") failed: Property not found. retry in 550.608856ms file = illumos-utils/src/svc.rs:36 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38 zone = oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38
29312025-01-10T22:08:00.859ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38") failed: Property not found. retry in 1.282644904s file = illumos-utils/src/svc.rs:36 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38 zone = oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38
29322025-01-10T22:08:02.154ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38") failed: Property not found. retry in 701.49157ms file = illumos-utils/src/svc.rs:36 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38 zone = oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38
29332025-01-10T22:08:02.867ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38") failed: Property not found. retry in 502.777488ms file = illumos-utils/src/svc.rs:36 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38 zone = oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38
29342025-01-10T22:08:03.380ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38") failed: Property not found. retry in 896.97299ms file = illumos-utils/src/svc.rs:36 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38 zone = oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38
29352025-01-10T22:08:04.306ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38 file = sled-agent/src/instance.rs:2067 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29362025-01-10T22:08:04.317ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38") failed: Property not found. retry in 27.680899ms file = illumos-utils/src/svc.rs:36 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29372025-01-10T22:08:04.358ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38") failed: Property not found. retry in 51.701148ms file = illumos-utils/src/svc.rs:36 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29382025-01-10T22:08:04.419ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38") failed: Property not found. retry in 291.676158ms file = illumos-utils/src/svc.rs:36 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29392025-01-10T22:08:04.720ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38") failed: Property not found. retry in 559.468661ms file = illumos-utils/src/svc.rs:36 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29402025-01-10T22:08:05.030ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47520
29412025-01-10T22:08:05.135ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34356
29422025-01-10T22:08:05.136ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 132 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34356 req_id = e10363c4-1000-4584-81de-1d93f1a8053e response_code = 200 uri = /v2p
29432025-01-10T22:08:05.149ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49202
29442025-01-10T22:08:05.149ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
29452025-01-10T22:08:05.191ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
29462025-01-10T22:08:05.253ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103732 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:49202 req_id = 018b73a3-7a21-4c6f-9bd3-b4dfacd2c53d response_code = 204 uri = /switch-ports
29472025-01-10T22:08:05.253ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 237 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49202 req_id = 1280a95e-1fcc-42b4-be7d-75fa5ec98e33 response_code = 200 uri = /network-bootstore-config
29482025-01-10T22:08:05.291ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:2076 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29492025-01-10T22:08:05.330ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:2100 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29502025-01-10T22:08:05.470ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:1213 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29512025-01-10T22:08:05.472ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1249 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
29522025-01-10T22:08:05.472ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Creating), migration_in: None, migration_out: None, time: 2025-01-10T22:08:05.472603232Z } file = sled-agent/src/instance.rs:780 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29532025-01-10T22:08:05.472ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2025-01-10T22:08:05.472603232Z } propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29542025-01-10T22:08:05.472ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 442570 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47520 req_id = 307261b1-33e9-4427-bb62-bcc0456cd903 response_code = 200 uri = /vmms/1aed9130-b669-4f4f-972c-06e21de1fc38/state
29552025-01-10T22:08:05.472ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7656056 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53958 req_id = cbe72f0f-93d6-4020-98ee-8024975aaf65 response_code = 204 uri = /eip-gateways
29562025-01-10T22:08:05.473ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38 state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2025-01-10T22:08:05.472603232Z }, migration_in: None, migration_out: None }
29572025-01-10T22:08:05.513ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_in: None, migration_out: None, time: 2025-01-10T22:08:05.513151509Z } file = sled-agent/src/instance.rs:780 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29582025-01-10T22:08:05.513ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-01-10T22:08:05.513151509Z } propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29592025-01-10T22:08:05.513ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2025-01-10T22:08:05.513151509Z }, migration_in: None, migration_out: None }
29602025-01-10T22:08:05.578ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 512 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53958 req_id = 56e8d205-c2f1-441a-850c-676a951d0d2d response_code = 204 uri = /vpc-routes
29612025-01-10T22:08:05.642ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35916
29622025-01-10T22:08:05.643ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35916 req_id = b866a3c3-1f79-4a34-87f3-3f2bfb9302de response_code = 200 uri = /vpc-routes
29632025-01-10T22:08:05.687ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {dad6653d-2a0a-4a4a-bb5e-32d25108dad3: {10.151.2.176: {29ac68e7-94b9-4881-adaa-290583c8a55e}, 10.151.2.175: {29ac68e7-94b9-4881-adaa-290583c8a55e}}} }
29642025-01-10T22:08:05.687ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35916 req_id = f167acb4-f958-4865-a880-448ec660cecb response_code = 204 uri = /eip-gateways
29652025-01-10T22:08:05.704ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 277 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35916 req_id = 1d937596-5508-421e-8523-80a261fae4c2 response_code = 204 uri = /vpc-routes
29662025-01-10T22:08:05.854ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57629
29672025-01-10T22:08:05.854ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 73 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57629 req_id = 6343d090-7fa2-499f-bb96-4f3df64ebc9e response_code = 200 uri = /vmms/1aed9130-b669-4f4f-972c-06e21de1fc38/state
29682025-01-10T22:08:05.977ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54933
29692025-01-10T22:08:05.978ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 50 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54933 req_id = cfac8927-94dd-473d-8702-b3009834265b response_code = 200 uri = /vpc-routes
29702025-01-10T22:08:06.035ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {dad6653d-2a0a-4a4a-bb5e-32d25108dad3: {10.151.2.176: {29ac68e7-94b9-4881-adaa-290583c8a55e}, 10.151.2.175: {29ac68e7-94b9-4881-adaa-290583c8a55e}}} }
29712025-01-10T22:08:06.035ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54933 req_id = ff122cd3-e134-45fd-8bf2-b7b819b3b80f response_code = 204 uri = /eip-gateways
29722025-01-10T22:08:06.051ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50936
29732025-01-10T22:08:06.051ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
29742025-01-10T22:08:06.088ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
29752025-01-10T22:08:06.145ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93750 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:50936 req_id = 555a9add-ac85-42df-9536-5fd3e3db22e0 response_code = 204 uri = /switch-ports
29762025-01-10T22:08:06.145ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49760
29772025-01-10T22:08:06.145ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39129
29782025-01-10T22:08:06.146ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 316 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50936 req_id = b368a8e6-1fc1-49e7-b87d-20bda2a33c8e response_code = 200 uri = /network-bootstore-config
29792025-01-10T22:08:06.146ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 278 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49760 req_id = e0a53bd1-fe22-4a18-b69d-7f067e0a236a response_code = 200 uri = /v2p
29802025-01-10T22:08:06.146ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 286 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39129 req_id = 1b97ac0e-2d36-4449-bd82-c24014f5f32d response_code = 200 uri = /vmms/1aed9130-b669-4f4f-972c-06e21de1fc38/state
29812025-01-10T22:08:06.146ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 629 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:54933 req_id = 1251a0e6-ebb7-436b-bf4f-12adccf416cc response_code = 204 uri = /vpc-routes
29822025-01-10T22:08:06.158ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52973
29832025-01-10T22:08:06.158ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52973 req_id = c2de2607-7ff1-449f-9f0f-56ab9a88769e response_code = 200 uri = /vpc-routes
29842025-01-10T22:08:06.177ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {dad6653d-2a0a-4a4a-bb5e-32d25108dad3: {10.151.2.175: {29ac68e7-94b9-4881-adaa-290583c8a55e}, 10.151.2.176: {29ac68e7-94b9-4881-adaa-290583c8a55e}}} }
29852025-01-10T22:08:06.177ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52973 req_id = 4dfb2502-ab8f-4a4e-9aae-bc46ba95c085 response_code = 204 uri = /eip-gateways
29862025-01-10T22:08:06.195ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 278 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52973 req_id = 90c956fe-52b2-4211-8bf9-9eeeb738a124 response_code = 204 uri = /vpc-routes
29872025-01-10T22:08:06.243ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:65129
29882025-01-10T22:08:06.243ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:65129 req_id = c6b3842c-7875-47a1-b32f-51e741cc9561 response_code = 200 uri = /v2p
29892025-01-10T22:08:06.270ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54698
29902025-01-10T22:08:06.270ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4145
29912025-01-10T22:08:06.308ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4169
29922025-01-10T22:08:06.364ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93225 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:54698 req_id = b62b41fc-46a7-4ba7-a7de-f55427aaf055 response_code = 204 uri = /switch-ports
29932025-01-10T22:08:06.364ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54698 req_id = c541623f-f58f-4152-941d-4ed74cac096d response_code = 200 uri = /network-bootstore-config
29942025-01-10T22:08:15.216ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49041
29952025-01-10T22:08:15.217ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 492 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:49041 req_id = c19f8055-f5ba-404f-b7b4-cd95e842b35b response_code = 200 uri = /vmms/1aed9130-b669-4f4f-972c-06e21de1fc38/state
29962025-01-10T22:08:15.217ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_in: None, migration_out: None, time: 2025-01-10T22:08:15.217653323Z } file = sled-agent/src/instance.rs:780 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29972025-01-10T22:08:15.217ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2025-01-10T22:08:15.217653323Z } propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
29982025-01-10T22:08:15.217ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38 state = SledVmmState { vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2025-01-10T22:08:15.217653323Z }, migration_in: None, migration_out: None }
29992025-01-10T22:08:15.268ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_in: None, migration_out: None, time: 2025-01-10T22:08:15.268357441Z } file = sled-agent/src/instance.rs:780 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
30002025-01-10T22:08:15.268ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2025-01-10T22:08:15.268357441Z } propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
30012025-01-10T22:08:15.268ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:816 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
30022025-01-10T22:08:15.268ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
30032025-01-10T22:08:15.268ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
30042025-01-10T22:08:15.268ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/10481327-13d1-4548-950f-130d063e2690/debug/bundle/zone", "/pool/int/53b46f94-5cb9-4bbb-a551-b5457accb3ee/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/73828a50-9586-442b-b526-668d3e1de808/crypt/debug", "/pool/ext/598e3879-d23d-4354-9e0c-045503f010a6/crypt/debug", "/pool/ext/5d7ad0a3-e47e-4369-b599-d02dbd3a8829/crypt/debug", "/pool/ext/d27ec511-4e48-4e26-9677-295de0452e91/crypt/debug", "/pool/ext/ce5c629d-252d-4e12-9f9d-2087cb772d81/crypt/debug", "/pool/ext/7b0e1b76-af4f-49bd-9848-611d8d8666ad/crypt/debug", "/pool/ext/619e5037-b196-43f6-b092-666b1b1387ff/crypt/debug", "/pool/ext/eecab578-d379-46e4-90f5-9dc11839dad5/crypt/debug", "/pool/ext/784c4a6b-3804-44fa-afd4-c974bef9dd5d/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:339 zone_name = oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38
30052025-01-10T22:08:15.902ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1142 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38", bundle_id: 0daa68c2-bf04-4644-9ac8-26ec5c674294 }, time_created: 2025-01-10T22:08:15.269112663Z, version: 0, cause: TerminatedInstance }
30062025-01-10T22:08:15.902ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_1aed9130-b669-4f4f-972c-06e21de1fc38 file = sled-agent/src/instance.rs:1313 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
30072025-01-10T22:08:17.665ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:279 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
30082025-01-10T22:08:17.687ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7808 local_addr = [fd00:1122:3344:101::1]:37589 method = GET remote_addr = [fd00:1122:3344:101::d]:40969 req_id = 51c8e28e-3456-41f7-927c-5ce2b50303c4 response_code = 200 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 uri = /2235d715-657e-4a92-8f59-9a6c0c957b73
30092025-01-10T22:08:17.688ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:1000 id = RouterId { vni: Vni(14455117), kind: System }
30102025-01-10T22:08:17.688ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:1000 id = RouterId { vni: Vni(14455117), kind: Custom(V4(Ipv4Net { addr: 172.30.0.0, width: 22 })) }
30112025-01-10T22:08:17.689ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-01-10T22:08:17.689563598Z }, migration_in: None, migration_out: None }
30122025-01-10T22:08:17.712ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2025-01-10T22:08:17.689563598Z }, migration_in: None, migration_out: None }
30132025-01-10T22:08:17.716ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:1250 instance_id = 191fa903-5a42-4777-b82a-12d4334e9b89 propolis_id = 1aed9130-b669-4f4f-972c-06e21de1fc38
30142025-01-10T22:08:18.159ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47780
30152025-01-10T22:08:18.159ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 50 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47780 req_id = dc7d6a02-eb47-4012-8119-2dca88517ce4 response_code = 200 uri = /vpc-routes
30162025-01-10T22:08:18.199ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
30172025-01-10T22:08:18.199ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1249 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73
30182025-01-10T22:08:18.199ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:47780 req_id = b56066c3-0c13-4491-9c6a-037e75fef401 response_code = 204 uri = /eip-gateways
30192025-01-10T22:08:18.207ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 149 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:47780 req_id = 29da88bb-c17f-4203-b661-52925fe7dd37 response_code = 204 uri = /vpc-routes
30202025-01-10T22:08:18.336ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34621
30212025-01-10T22:08:18.336ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 132 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34621 req_id = 5c163c25-07fb-42bd-bb2a-21526e4a4a92 response_code = 200 uri = /v2p
30222025-01-10T22:08:18.483ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41701
30232025-01-10T22:08:18.483ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 140 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41701 req_id = bcec67c6-fa0e-42cb-8bef-fad6e7050c63 response_code = 200 uri = /v2p
30242025-01-10T22:08:18.584ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48944
30252025-01-10T22:08:18.584ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48944 req_id = 280cbce8-e542-4097-a0d3-41860234e8f1 response_code = 200 uri = /vpc-routes
30262025-01-10T22:08:18.624ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
30272025-01-10T22:08:18.624ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48944 req_id = 5931136f-f2ee-4dc5-bf72-b2ac7624aa6d response_code = 204 uri = /eip-gateways
30282025-01-10T22:08:18.639ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 172 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48944 req_id = eb05e61d-2091-4d92-823f-c97403f30709 response_code = 204 uri = /vpc-routes
30292025-01-10T22:08:19.233ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33191
30302025-01-10T22:08:19.233ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33191 req_id = ca5c5be0-cbc7-4512-8d32-b3b2c4745087 response_code = 200 uri = /vpc-routes
30312025-01-10T22:08:19.246ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = 2235d715-657e-4a92-8f59-9a6c0c957b73 values = ExternalIpGatewayMap { mappings: {} }
30322025-01-10T22:08:19.246ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 66 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:33191 req_id = 5bed8346-9145-454b-998a-b02b83144279 response_code = 204 uri = /eip-gateways
30332025-01-10T22:08:19.253ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 207 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:33191 req_id = 8892e12d-14e7-474e-940b-fc5d0244d380 response_code = 204 uri = /vpc-routes