Max level shown:
1 [ Dec 21 01:14:06 Enabled. ]
2 [ Dec 21 01:14:06 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Dec 21 01:14:06 Method "start" exited with status 0. ]
4 [ Dec 21 01:14:06 Rereading configuration. ]
5 [ Dec 21 01:14:06 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72024-12-21T01:14:06.385ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:113
82024-12-21T01:14:06.386ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:255
92024-12-21T01:14:06.440ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:262
102024-12-21T01:14:06.440ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
112024-12-21T01:14:06.555ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
122024-12-21T01:14:06.563ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
132024-12-21T01:14:06.572ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:272 path = /var/run/oxide/
142024-12-21T01:14:07.176ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fdb0:18c0:4d0d:a02a::, width: 64 }
152024-12-21T01:14:07.316ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:126
162024-12-21T01:14:07.316ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:138
172024-12-21T01:14:07.316ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:151
182024-12-21T01:14:07.316ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:173 nongimlet_observed_disks = [] sled_mode = Scrimlet { asic: SoftNpuZone }
192024-12-21T01:14:07.316ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:717
202024-12-21T01:14:07.316ZINFOSledAgent (DumpSetup-worker): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:520
212024-12-21T01:14:07.319ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "sock", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:624
222024-12-21T01:14:07.319ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:766
232024-12-21T01:14:07.319ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:187
242024-12-21T01:14:07.319ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:825
252024-12-21T01:14:07.319ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_0.vdev
262024-12-21T01:14:07.319ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = None
272024-12-21T01:14:07.319ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
282024-12-21T01:14:07.319ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
292024-12-21T01:14:07.319ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } path = /scratch/m2_0.vdev
302024-12-21T01:14:07.331ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_0.vdev file = sled-hardware/src/disk.rs:367
312024-12-21T01:14:07.331ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = 39f3a11f-ebbe-4984-ad03-60e64a112dfb (zpool)
322024-12-21T01:14:07.429ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 39f3a11f-ebbe-4984-ad03-60e64a112dfb (zpool), kind: Internal }
332024-12-21T01:14:07.799ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:607
342024-12-21T01:14:07.799ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_1.vdev
352024-12-21T01:14:07.799ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
362024-12-21T01:14:07.799ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
372024-12-21T01:14:07.799ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
382024-12-21T01:14:07.799ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
392024-12-21T01:14:07.799ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } path = /scratch/m2_1.vdev
402024-12-21T01:14:07.808ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_1.vdev file = sled-hardware/src/disk.rs:367
412024-12-21T01:14:07.808ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = 1bf7a526-12df-41e0-8b64-89b419ddc8ee (zpool)
422024-12-21T01:14:07.897ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 1bf7a526-12df-41e0-8b64-89b419ddc8ee (zpool), kind: Internal }
432024-12-21T01:14:08.267ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_0.vdev
442024-12-21T01:14:08.267ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 39f3a11f-ebbe-4984-ad03-60e64a112dfb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 1bf7a526-12df-41e0-8b64-89b419ddc8ee (zpool), kind: Internal } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
452024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
462024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
472024-12-21T01:14:08.267ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
482024-12-21T01:14:08.267ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
492024-12-21T01:14:08.267ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_1.vdev
502024-12-21T01:14:08.267ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 39f3a11f-ebbe-4984-ad03-60e64a112dfb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 1bf7a526-12df-41e0-8b64-89b419ddc8ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
512024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
522024-12-21T01:14:08.267ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
532024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
542024-12-21T01:14:08.267ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
552024-12-21T01:14:08.267ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_2.vdev
562024-12-21T01:14:08.267ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
572024-12-21T01:14:08.267ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 39f3a11f-ebbe-4984-ad03-60e64a112dfb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 1bf7a526-12df-41e0-8b64-89b419ddc8ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
582024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
592024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
602024-12-21T01:14:08.267ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
612024-12-21T01:14:08.267ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_3.vdev
622024-12-21T01:14:08.267ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 39f3a11f-ebbe-4984-ad03-60e64a112dfb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 1bf7a526-12df-41e0-8b64-89b419ddc8ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
632024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
642024-12-21T01:14:08.267ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
652024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
662024-12-21T01:14:08.267ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
672024-12-21T01:14:08.267ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_4.vdev
682024-12-21T01:14:08.267ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 39f3a11f-ebbe-4984-ad03-60e64a112dfb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 1bf7a526-12df-41e0-8b64-89b419ddc8ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
692024-12-21T01:14:08.267ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
702024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
712024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
722024-12-21T01:14:08.267ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
732024-12-21T01:14:08.267ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_5.vdev
742024-12-21T01:14:08.267ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 39f3a11f-ebbe-4984-ad03-60e64a112dfb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 1bf7a526-12df-41e0-8b64-89b419ddc8ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
752024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
762024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
772024-12-21T01:14:08.267ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
782024-12-21T01:14:08.267ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
792024-12-21T01:14:08.267ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_6.vdev
802024-12-21T01:14:08.267ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 39f3a11f-ebbe-4984-ad03-60e64a112dfb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 1bf7a526-12df-41e0-8b64-89b419ddc8ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
812024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
822024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
832024-12-21T01:14:08.267ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
842024-12-21T01:14:08.267ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
852024-12-21T01:14:08.267ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_7.vdev
862024-12-21T01:14:08.267ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
872024-12-21T01:14:08.267ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
882024-12-21T01:14:08.267ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_8.vdev
892024-12-21T01:14:08.267ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 39f3a11f-ebbe-4984-ad03-60e64a112dfb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 1bf7a526-12df-41e0-8b64-89b419ddc8ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
902024-12-21T01:14:08.267ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:607
912024-12-21T01:14:08.267ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
922024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
932024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
942024-12-21T01:14:08.267ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:97
952024-12-21T01:14:08.267ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-agent/src/long_running_tasks.rs:99
962024-12-21T01:14:08.267ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 39f3a11f-ebbe-4984-ad03-60e64a112dfb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 1bf7a526-12df-41e0-8b64-89b419ddc8ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
972024-12-21T01:14:08.267ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
982024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
992024-12-21T01:14:08.267ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1002024-12-21T01:14:08.267ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 39f3a11f-ebbe-4984-ad03-60e64a112dfb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 1bf7a526-12df-41e0-8b64-89b419ddc8ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
1012024-12-21T01:14:08.268ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
1022024-12-21T01:14:08.268ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1032024-12-21T01:14:08.268ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:211
1042024-12-21T01:14:08.268ZINFOSledAgent (bootstore): No ledger in /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1052024-12-21T01:14:08.268ZINFOSledAgent (bootstore): No ledger in /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1062024-12-21T01:14:08.268ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-sock-i86pc
1072024-12-21T01:14:08.268ZINFOSledAgent (bootstore): No ledger in /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1082024-12-21T01:14:08.268ZINFOSledAgent (bootstore): No ledger in /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1092024-12-21T01:14:08.268ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-sock-i86pc
1102024-12-21T01:14:08.268ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:216
1112024-12-21T01:14:08.268ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:231
1122024-12-21T01:14:08.278ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:737
1132024-12-21T01:14:08.278ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 1335, tv_nsec: 190846936 } time_to_next_cleanup = 599.999925899s
1142024-12-21T01:14:08.279ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3962
1152024-12-21T01:14:08.279ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
1162024-12-21T01:14:08.279ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/sled-agent-request.json file = common/src/ledger.rs:188
1172024-12-21T01:14:08.279ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/sled-agent-request.json file = common/src/ledger.rs:188
1182024-12-21T01:14:08.282ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fdb0:18c0:4d0d:a02a::1]:80
1192024-12-21T01:14:08.284ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:51 local_addr = [fdb0:18c0:4d0d:a02a::1]:12346
1202024-12-21T01:14:08.284ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, rib_priority: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: Some( None, ), bgp_peers: [], autoneg: false, lldp: None, tx_eq: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, } file = sled-agent/src/rack_setup/service.rs:1203
1212024-12-21T01:14:08.284ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1222024-12-21T01:14:08.284ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
1232024-12-21T01:14:08.284ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
1242024-12-21T01:14:08.284ZINFOSledAgent (RSS): No ledger in /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/rss-started.marker file = common/src/ledger.rs:188
1252024-12-21T01:14:08.284ZINFOSledAgent (RSS): No ledger in /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/rss-started.marker file = common/src/ledger.rs:188
1262024-12-21T01:14:08.284ZINFOSledAgent (RSS): No ledger in /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1272024-12-21T01:14:08.284ZINFOSledAgent (RSS): No ledger in /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1282024-12-21T01:14:08.284ZINFOSledAgent (RSS): RSS not previously run. Creating plans. file = sled-agent/src/rack_setup/service.rs:1252
1292024-12-21T01:14:08.284ZINFOSledAgent (RSS): Writing ledger to /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/.rss-started.marker.tmp file = common/src/ledger.rs:199
1302024-12-21T01:14:08.285ZINFOSledAgent (RSS): Writing ledger to /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/.rss-started.marker.tmp file = common/src/ledger.rs:199
1312024-12-21T01:14:08.285ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:1284
1322024-12-21T01:14:08.285ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:66
1332024-12-21T01:14:08.285ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:a02a::1 file = sled-agent/src/rack_setup/plan/sled.rs:39
1342024-12-21T01:14:08.285ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:1316
1352024-12-21T01:14:08.285ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
1362024-12-21T01:14:08.285ZINFOSledAgent (bootstore): Writing ledger to /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1372024-12-21T01:14:08.285ZINFOSledAgent (bootstore): Writing ledger to /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1382024-12-21T01:14:08.285ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
1392024-12-21T01:14:08.285ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:232 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: b1eebc13-191d-4a5d-a35e-84ae64fecac9 (sled), rack_id: 18f9683c-e647-4a58-a294-698fb36fa534, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
1402024-12-21T01:14:08.286ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:4259
1412024-12-21T01:14:08.286ZINFOSledAgent (ServiceManager): Starting switch zone file = sled-agent/src/services.rs:4659
1422024-12-21T01:14:08.290ZINFOSledAgent (SledAgentSprocketsClient): Loaded keys and certs BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/client.rs:50
1432024-12-21T01:14:08.296ZINFOSledAgent (SledAgentSprocketsClient): Certificate chain verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1442024-12-21T01:14:08.296ZINFOSledAgent (SledAgentSprocketsClient): Signature verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1452024-12-21T01:14:08.300ZINFOSledAgent (SledAgentSprocketsServer): Certificate chain verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1462024-12-21T01:14:08.300ZINFOSledAgent (SledAgentSprocketsServer): Signature verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1472024-12-21T01:14:08.301ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:74 remote_addr = [fdb0:18c0:4d0d:a02a::1]:62521
1482024-12-21T01:14:08.301ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: b1eebc13-191d-4a5d-a35e-84ae64fecac9 (sled), rack_id: 18f9683c-e647-4a58-a294-698fb36fa534, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } file = sled-agent/src/bootstrap/server.rs:362
1492024-12-21T01:14:08.301ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:377
1502024-12-21T01:14:08.301ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:44
1512024-12-21T01:14:08.301ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1522024-12-21T01:14:08.301ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:607
1532024-12-21T01:14:08.301ZINFOSledAgent (StorageManager): Loading storage config file = sled-storage/src/manager.rs:813
1542024-12-21T01:14:08.301ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:101::, width: 64 }
1552024-12-21T01:14:08.301ZINFOSledAgent (StorageManager): No ledger in /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1562024-12-21T01:14:08.301ZINFOSledAgent (StorageManager): No ledger in /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1572024-12-21T01:14:08.301ZINFOSledAgent (StorageManager): No ledger of physical disks exists file = sled-storage/src/manager.rs:802
1582024-12-21T01:14:08.301ZINFOSledAgent (StorageManager): KeyManager ready, but no ledger detected file = sled-storage/src/manager.rs:830
1592024-12-21T01:14:08.301ZINFOSledAgent (StorageManager): automatically managing disks file = sled-storage/src/manager.rs:596
1602024-12-21T01:14:08.301ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
1612024-12-21T01:14:08.301ZINFOSledAgent (StorageManager): Successfully synchronized disks without error file = sled-storage/src/manager.rs:701 result = DisksManagementResult { status: [] }
1622024-12-21T01:14:08.316ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
1632024-12-21T01:14:08.336ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:325
1642024-12-21T01:14:08.352ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:363
1652024-12-21T01:14:08.365ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:412 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
1662024-12-21T01:14:08.365ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
1672024-12-21T01:14:08.365ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:424 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
1682024-12-21T01:14:08.365ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1692024-12-21T01:14:08.368ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 87.896357ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "c37d9fc6-0848-43c3-87db-b1ff7a823c4b", "content-length": "124", "date": "Sat, 21 Dec 2024 01:14:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "c37d9fc6-0848-43c3-87db-b1ff7a823c4b" } file = clients/ddm-admin-client/src/lib.rs:177
1702024-12-21T01:14:08.379ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_39f3a11f-ebbe-4984-ad03-60e64a112dfb/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1712024-12-21T01:14:08.403ZINFOSledAgent: successfully created encrypted zvol: name="oxi_39f3a11f-ebbe-4984-ad03-60e64a112dfb/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1722024-12-21T01:14:08.403ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_39f3a11f-ebbe-4984-ad03-60e64a112dfb/swap" file = sled-agent/src/swap_device.rs:96
1732024-12-21T01:14:08.408ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:439 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
1742024-12-21T01:14:08.408ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:129
1752024-12-21T01:14:08.408ZINFOSledAgent (BackingFs): Ensuring dataset oxi_39f3a11f-ebbe-4984-ad03-60e64a112dfb/backing/fmd file = sled-agent/src/backing_fs.rs:139
1762024-12-21T01:14:08.457ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1772024-12-21T01:14:08.457ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 166.814628ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "b143f86b-47e3-4712-8d4a-e69994b7d81a", "content-length": "124", "date": "Sat, 21 Dec 2024 01:14:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "b143f86b-47e3-4712-8d4a-e69994b7d81a" } file = clients/ddm-admin-client/src/lib.rs:177
1782024-12-21T01:14:08.478ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:174
1792024-12-21T01:14:08.625ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1802024-12-21T01:14:08.626ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 213.624273ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "6f729bca-f88d-40b5-93e2-fb4e778177be", "content-length": "124", "date": "Sat, 21 Dec 2024 01:14:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "6f729bca-f88d-40b5-93e2-fb4e778177be" } file = clients/ddm-admin-client/src/lib.rs:177
1812024-12-21T01:14:08.696ZINFOSledAgent (BackingFs): Mounting oxi_39f3a11f-ebbe-4984-ad03-60e64a112dfb/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:182
1822024-12-21T01:14:08.708ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:190
1832024-12-21T01:14:08.708ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:190
1842024-12-21T01:14:08.709ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:190
1852024-12-21T01:14:08.709ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:196
1862024-12-21T01:14:08.841ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1872024-12-21T01:14:08.841ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 165.971296ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "40ae52e2-3fa0-4f46-9df5-b0a52c900f57", "content-length": "124", "date": "Sat, 21 Dec 2024 01:14:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "40ae52e2-3fa0-4f46-9df5-b0a52c900f57" } file = clients/ddm-admin-client/src/lib.rs:177
1882024-12-21T01:14:09.008ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1892024-12-21T01:14:09.009ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 271.414986ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "ee7687a8-78cd-4b97-a4fb-9cb417e32076", "content-length": "124", "date": "Sat, 21 Dec 2024 01:14:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "ee7687a8-78cd-4b97-a4fb-9cb417e32076" } file = clients/ddm-admin-client/src/lib.rs:177
1902024-12-21T01:14:09.281ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1912024-12-21T01:14:09.282ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 366.309513ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "aea73bb1-36da-4181-8fd2-50582529dbed", "content-length": "124", "date": "Sat, 21 Dec 2024 01:14:08 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "aea73bb1-36da-4181-8fd2-50582529dbed" } file = clients/ddm-admin-client/src/lib.rs:177
1922024-12-21T01:14:09.446ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:93 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
1932024-12-21T01:14:09.452ZINFOSledAgent (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:42313 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
1942024-12-21T01:14:09.452ZINFOSledAgent (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
1952024-12-21T01:14:09.452ZINFOSledAgent (producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::1]:42313 file = oximeter/producer/src/lib.rs:279 interval = 30s producer_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
1962024-12-21T01:14:09.453ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 204.675316ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
1972024-12-21T01:14:09.454ZINFOSledAgent (VmmReservoirManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/vmm_reservoir.rs:267 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
1982024-12-21T01:14:09.454ZINFOSledAgent (VmmReservoirManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:275 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
1992024-12-21T01:14:09.454ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:921 underlay_address = fd00:1122:3344:101::1
2002024-12-21T01:14:09.454ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
2012024-12-21T01:14:09.648ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
2022024-12-21T01:14:09.659ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 714.657084ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
2032024-12-21T01:14:10.375ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 1.301826868s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
2042024-12-21T01:14:11.678ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 2.575619904s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
2052024-12-21T01:14:12.068ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2683
2062024-12-21T01:14:12.068ZINFOSledAgent (ServiceManager): Setting up lldpd service file = sled-agent/src/services.rs:2928
2072024-12-21T01:14:12.068ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2553
2082024-12-21T01:14:12.068ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2600
2092024-12-21T01:14:12.068ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:3009
2102024-12-21T01:14:12.068ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:3054
2112024-12-21T01:14:12.070ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2853
2122024-12-21T01:14:12.071ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2594
2132024-12-21T01:14:12.072ZINFOSledAgent (ServiceManager): Profile for oxz_switch: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/mgs"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12225'/> <propval type="astring" name="id" value='a33da4ba-af6c-4274-ab44-51d63abaafb0'/> <propval type="astring" name="rack_id" value='18f9683c-e647-4a58-a294-698fb36fa534'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/wicketd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12226'/> <propval type="astring" name="artifact-address" value='[fdb0:18c0:4d0d:a02a::2]:12227'/> <propval type="astring" name="baseboard-file" value='/opt/oxide/baseboard.json'/> <propval type="astring" name="mgs-address" value='[::1]:12225'/> <propval type="astring" name="nexus-proxy-address" value='[::]:12229'/> <propval type="astring" name="rack-subnet" value='fd00:1122:3344::'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/switch_zone_setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="gz_local_link_addr" value='fe80::8:20ff:fe9e:4ff'/> <property type="astring" name="link_local_links"> <astring_list> <value_node value="tfportrear0_0" /> <value_node value="oxBootstrap0" /> </astring_list> </property> <propval type="astring" name="bootstrap_addr" value='fdb0:18c0:4d0d:a02a::2'/> <propval type="astring" name="bootstrap_vnic" value='oxBootstrap0'/> <propval type="astring" name="baseboard_info" value='{"type":"pc","identifier":"sock","model":"i86pc"}'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/dendrite"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="rack_id" value='18f9683c-e647-4a58-a294-698fb36fa534'/> <propval type="astring" name="sled_id" value='b1eebc13-191d-4a5d-a35e-84ae64fecac9'/> <propval type="astring" name="sled_model" value='i86pc'/> <propval type="astring" name="sled_serial" value='sock'/> <propval type="astring" name="sled_revision" value='0'/> <propval type="astring" name="address" value='[::1]:12224'/> <propval type="astring" name="front_ports" value='1'/> <propval type="astring" name="rear_ports" value='1'/> <propval type="astring" name="port_config" value='/opt/oxide/dendrite/misc/softnpu_single_sled_config.toml'/> <propval type="astring" name="mgmt" value='uds'/> <propval type="astring" name="uds_path" value='/opt/softnpu/stuff'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/tfport"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="host" value='[::1]'/> <propval type="astring" name="port" value='12224'/> <propval type="astring" name="flags" value='--sync-only'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/lldpd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="board_rev" value='softnpu_front_1_rear_1'/> <propval type="astring" name="scrimlet_id" value='sock'/> <propval type="astring" name="scrimlet_model" value='i86pc'/> <propval type="astring" name="address" value='[::1]:12230'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/pumpkind"> </service> <service version="1" type="service" name="oxide/mgd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_uuid" value='b1eebc13-191d-4a5d-a35e-84ae64fecac9'/> <propval type="astring" name="rack_uuid" value='18f9683c-e647-4a58-a294-698fb36fa534'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/mg-ddm"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="mode" value='transit'/> <propval type="astring" name="dendrite" value='true'/> <propval type="astring" name="sled_uuid" value='b1eebc13-191d-4a5d-a35e-84ae64fecac9'/> <propval type="astring" name="rack_uuid" value='18f9683c-e647-4a58-a294-698fb36fa534'/> <propval type="astring" name="interfaces" value='tfportrear0_0/ll'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/uplink"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
2142024-12-21T01:14:12.072ZINFOSledAgent (ServiceManager): Booting oxz_switch zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_switch
2152024-12-21T01:14:12.380ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 71.184364ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2162024-12-21T01:14:12.606ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 94.09236ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2172024-12-21T01:14:12.800ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 263.328158ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2182024-12-21T01:14:13.185ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 280.358252ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2192024-12-21T01:14:13.602ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.188512462s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2202024-12-21T01:14:14.256ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 3.013537412s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
2212024-12-21T01:14:14.839ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.371884396s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2222024-12-21T01:14:16.276ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 523.94859ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2232024-12-21T01:14:16.974ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 830.987108ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2242024-12-21T01:14:17.272ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 10.32550504s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
2252024-12-21T01:14:18.131ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/b1eebc13-191d-4a5d-a35e-84ae64fecac9) file = sled-agent/src/nexus.rs:527 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
2262024-12-21T01:14:18.162ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
2272024-12-21T01:14:18.162ZINFOSledAgent (dropshot (Repo Depot)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:12348 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
2282024-12-21T01:14:18.163ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:12345
2292024-12-21T01:14:18.163ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:432
2302024-12-21T01:14:18.163ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
2312024-12-21T01:14:18.163ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2322024-12-21T01:14:18.163ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2332024-12-21T01:14:18.164ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:138
2342024-12-21T01:14:18.164ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56 }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [PortConfigV2 { routes: [RouteConfig { destination: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), nexthop: 10.151.2.100, vlan_id: None, rib_priority: None }], addresses: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: Some(None), bgp_peers: [], autoneg: false, lldp: None, tx_eq: None }], bgp: [], bfd: [] })))
2352024-12-21T01:14:18.164ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3962
2362024-12-21T01:14:18.164ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:81 remote_addr = [fdb0:18c0:4d0d:a02a::1]:62521
2372024-12-21T01:14:18.350ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:117 peer = [fdb0:18c0:4d0d:a02a::1]:12346
2382024-12-21T01:14:18.350ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:252 target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
2392024-12-21T01:14:18.355ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:4283 new = [fd00:1122:3344:101::2, ::1] old = [::1]
2402024-12-21T01:14:18.355ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists file = sled-agent/src/services.rs:4299
2412024-12-21T01:14:18.355ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::2, prefix: 64 })) file = illumos-utils/src/running_zone.rs:534 zone = oxz_switch
2422024-12-21T01:14:18.623ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55559
2432024-12-21T01:14:18.623ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
2442024-12-21T01:14:18.623ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
2452024-12-21T01:14:18.623ZINFOSledAgent (ServiceManager): No ledger in /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-zones.json file = common/src/ledger.rs:188
2462024-12-21T01:14:18.623ZINFOSledAgent (ServiceManager): No ledger in /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-zones.json file = common/src/ledger.rs:188
2472024-12-21T01:14:18.623ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 295 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:55559 req_id = 521ea41d-056f-4497-8cd1-edca8c8b5b92 response_code = 200 uri = /inventory
2482024-12-21T01:14:18.653ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:62773
2492024-12-21T01:14:18.653ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 50 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:62773 req_id = bec0a1ff-fabe-4080-a6c8-2408cd1cec02 response_code = 200 uri = /sled-role
2502024-12-21T01:14:18.685ZINFOSledAgent (RSS): attempting to set up sled's storage: OmicronPhysicalDisksConfig { generation: Generation(2), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 4fbe2f3a-ff24-4fcf-a039-4fe6ae723a08 (physical_disk), pool_id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: 813cf0b0-9211-4ce9-abf8-7262c71a3202 (physical_disk), pool_id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 4fa69ccd-6472-4c6f-b696-3fedcb6ba1e7 (physical_disk), pool_id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 20801814-1fa9-49b2-8fa0-5457bee29c65 (physical_disk), pool_id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 4e5b2d57-e1f4-4bce-91e9-4f4742691715 (physical_disk), pool_id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 1bd419dd-484a-48c1-b18c-51ec934ff14f (physical_disk), pool_id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: f2e132ac-4246-456e-a3a8-95584be9efd2 (physical_disk), pool_id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: a5cc2686-d4b8-4047-8318-8e5ec743efb3 (physical_disk), pool_id: 34789575-52b6-411b-9614-90f11629cb04 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 0b88ed69-13bc-4126-bf2a-789b88d0aa57 (physical_disk), pool_id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool) }] } file = sled-agent/src/rack_setup/service.rs:385 sled_address = [fd00:1122:3344:101::1]:12345
2512024-12-21T01:14:18.685ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:55110
2522024-12-21T01:14:18.685ZINFOSledAgent: physical disks ensure file = sled-agent/src/sled_agent.rs:893 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
2532024-12-21T01:14:18.685ZINFOSledAgent (StorageManager): Received OmicronPhysicalDisksEnsure { config: OmicronPhysicalDisksConfig { generation: Generation(2), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: 4fbe2f3a-ff24-4fcf-a039-4fe6ae723a08 (physical_disk), pool_id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: 813cf0b0-9211-4ce9-abf8-7262c71a3202 (physical_disk), pool_id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: 4fa69ccd-6472-4c6f-b696-3fedcb6ba1e7 (physical_disk), pool_id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: 20801814-1fa9-49b2-8fa0-5457bee29c65 (physical_disk), pool_id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 4e5b2d57-e1f4-4bce-91e9-4f4742691715 (physical_disk), pool_id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: 1bd419dd-484a-48c1-b18c-51ec934ff14f (physical_disk), pool_id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: f2e132ac-4246-456e-a3a8-95584be9efd2 (physical_disk), pool_id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: a5cc2686-d4b8-4047-8318-8e5ec743efb3 (physical_disk), pool_id: 34789575-52b6-411b-9614-90f11629cb04 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 0b88ed69-13bc-4126-bf2a-789b88d0aa57 (physical_disk), pool_id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool) }] }, tx: ... } file = sled-storage/src/manager.rs:607
2542024-12-21T01:14:18.685ZINFOSledAgent (StorageManager): No ledger in /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2552024-12-21T01:14:18.685ZINFOSledAgent (StorageManager): No ledger in /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2562024-12-21T01:14:18.685ZINFOSledAgent (StorageManager): No previously-stored 'requested disks', creating new ledger file = sled-storage/src/manager.rs:1298 request = omicron_physical_disks_ensure
2572024-12-21T01:14:18.685ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
2582024-12-21T01:14:18.685ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:342
2592024-12-21T01:14:18.685ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2602024-12-21T01:14:18.685ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } path = /scratch/u2_0.vdev
2612024-12-21T01:14:18.846ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_0.vdev file = sled-hardware/src/disk.rs:367
2622024-12-21T01:14:18.846ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool)
2632024-12-21T01:14:19.272ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/b1eebc13-191d-4a5d-a35e-84ae64fecac9) file = sled-agent/src/probe_manager.rs:231 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
2642024-12-21T01:14:19.500ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 11062531147977962647 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
2652024-12-21T01:14:19.500ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 14223862254712097596 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
2662024-12-21T01:14:19.691ZINFOSledAgent (VmmReservoirManager): Finished setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:279 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
2672024-12-21T01:14:19.721ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }
2682024-12-21T01:14:19.730ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:239
2692024-12-21T01:14:19.730ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:241
2702024-12-21T01:14:19.730ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:245
2712024-12-21T01:14:19.730ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:247
2722024-12-21T01:14:19.730ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2732024-12-21T01:14:19.731ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2742024-12-21T01:14:19.731ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
2752024-12-21T01:14:19.767ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2762024-12-21T01:14:19.776ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/zone file = sled-storage/src/dataset.rs:302
2772024-12-21T01:14:19.915ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff file = sled-storage/src/dataset.rs:388
2782024-12-21T01:14:19.923ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:440
2792024-12-21T01:14:19.923ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:357
2802024-12-21T01:14:19.923ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:342
2812024-12-21T01:14:19.923ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2822024-12-21T01:14:19.923ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } path = /scratch/u2_1.vdev
2832024-12-21T01:14:19.933ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_1.vdev file = sled-hardware/src/disk.rs:367
2842024-12-21T01:14:19.933ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool)
2852024-12-21T01:14:20.038ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }
2862024-12-21T01:14:20.046ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:239
2872024-12-21T01:14:20.046ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:241
2882024-12-21T01:14:20.046ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:245
2892024-12-21T01:14:20.046ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:247
2902024-12-21T01:14:20.046ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2912024-12-21T01:14:20.046ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2922024-12-21T01:14:20.046ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
2932024-12-21T01:14:20.082ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2942024-12-21T01:14:20.090ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/zone file = sled-storage/src/dataset.rs:302
2952024-12-21T01:14:20.230ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_41cc83ed-adbe-492f-ae31-7074b31692b0 file = sled-storage/src/dataset.rs:388
2962024-12-21T01:14:20.239ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:440
2972024-12-21T01:14:20.239ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:357
2982024-12-21T01:14:20.239ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:342
2992024-12-21T01:14:20.239ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3002024-12-21T01:14:20.239ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } path = /scratch/u2_2.vdev
3012024-12-21T01:14:20.248ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_2.vdev file = sled-hardware/src/disk.rs:367
3022024-12-21T01:14:20.249ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool)
3032024-12-21T01:14:20.345ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }
3042024-12-21T01:14:20.353ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:239
3052024-12-21T01:14:20.353ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:241
3062024-12-21T01:14:20.353ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:245
3072024-12-21T01:14:20.353ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:247
3082024-12-21T01:14:20.353ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3092024-12-21T01:14:20.353ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3102024-12-21T01:14:20.353ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
3112024-12-21T01:14:20.357ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/b1eebc13-191d-4a5d-a35e-84ae64fecac9) file = sled-agent/src/probe_manager.rs:231 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
3122024-12-21T01:14:20.389ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3132024-12-21T01:14:20.397ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_7e02d0c4-4e5d-440d-94c1-593a67af3118/crypt/zone file = sled-storage/src/dataset.rs:302
3142024-12-21T01:14:20.537ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_7e02d0c4-4e5d-440d-94c1-593a67af3118 file = sled-storage/src/dataset.rs:388
3152024-12-21T01:14:20.545ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:440
3162024-12-21T01:14:20.545ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:357
3172024-12-21T01:14:20.545ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:342
3182024-12-21T01:14:20.545ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3192024-12-21T01:14:20.545ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } path = /scratch/u2_3.vdev
3202024-12-21T01:14:20.555ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_3.vdev file = sled-hardware/src/disk.rs:367
3212024-12-21T01:14:20.555ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool)
3222024-12-21T01:14:20.648ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }
3232024-12-21T01:14:20.656ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:239
3242024-12-21T01:14:20.656ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:241
3252024-12-21T01:14:20.656ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:245
3262024-12-21T01:14:20.656ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:247
3272024-12-21T01:14:20.656ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3282024-12-21T01:14:20.656ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3292024-12-21T01:14:20.656ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
3302024-12-21T01:14:20.691ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3312024-12-21T01:14:20.699ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crypt/zone file = sled-storage/src/dataset.rs:302
3322024-12-21T01:14:20.839ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50 file = sled-storage/src/dataset.rs:388
3332024-12-21T01:14:20.847ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:440
3342024-12-21T01:14:20.847ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:357
3352024-12-21T01:14:20.847ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:342
3362024-12-21T01:14:20.847ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3372024-12-21T01:14:20.847ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } path = /scratch/u2_4.vdev
3382024-12-21T01:14:20.857ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_4.vdev file = sled-hardware/src/disk.rs:367
3392024-12-21T01:14:20.857ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool)
3402024-12-21T01:14:20.948ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }
3412024-12-21T01:14:20.956ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:239
3422024-12-21T01:14:20.956ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:241
3432024-12-21T01:14:20.956ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:245
3442024-12-21T01:14:20.956ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:247
3452024-12-21T01:14:20.956ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3462024-12-21T01:14:20.956ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3472024-12-21T01:14:20.956ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
3482024-12-21T01:14:20.990ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3492024-12-21T01:14:20.998ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_10325d85-ce1b-49aa-8039-7e9482ec8803/crypt/zone file = sled-storage/src/dataset.rs:302
3502024-12-21T01:14:21.138ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_10325d85-ce1b-49aa-8039-7e9482ec8803 file = sled-storage/src/dataset.rs:388
3512024-12-21T01:14:21.146ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:440
3522024-12-21T01:14:21.146ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:357
3532024-12-21T01:14:21.146ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:342
3542024-12-21T01:14:21.146ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3552024-12-21T01:14:21.146ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } path = /scratch/u2_5.vdev
3562024-12-21T01:14:21.155ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_5.vdev file = sled-hardware/src/disk.rs:367
3572024-12-21T01:14:21.155ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool)
3582024-12-21T01:14:21.250ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }
3592024-12-21T01:14:21.259ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:239
3602024-12-21T01:14:21.259ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:241
3612024-12-21T01:14:21.259ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:245
3622024-12-21T01:14:21.259ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:247
3632024-12-21T01:14:21.259ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3642024-12-21T01:14:21.259ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3652024-12-21T01:14:21.259ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
3662024-12-21T01:14:21.293ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3672024-12-21T01:14:21.301ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739/crypt/zone file = sled-storage/src/dataset.rs:302
3682024-12-21T01:14:21.359ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/b1eebc13-191d-4a5d-a35e-84ae64fecac9) file = sled-agent/src/probe_manager.rs:231 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
3692024-12-21T01:14:21.435ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 file = sled-storage/src/dataset.rs:388
3702024-12-21T01:14:21.443ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:440
3712024-12-21T01:14:21.443ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:357
3722024-12-21T01:14:21.443ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:342
3732024-12-21T01:14:21.443ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3742024-12-21T01:14:21.443ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } path = /scratch/u2_6.vdev
3752024-12-21T01:14:21.453ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_6.vdev file = sled-hardware/src/disk.rs:367
3762024-12-21T01:14:21.453ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool)
3772024-12-21T01:14:21.546ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }
3782024-12-21T01:14:21.554ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:239
3792024-12-21T01:14:21.554ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:241
3802024-12-21T01:14:21.554ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:245
3812024-12-21T01:14:21.554ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:247
3822024-12-21T01:14:21.554ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3832024-12-21T01:14:21.555ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3842024-12-21T01:14:21.555ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
3852024-12-21T01:14:21.588ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3862024-12-21T01:14:21.596ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_192cce10-69a8-4ed6-a459-9ca86a7206ec/crypt/zone file = sled-storage/src/dataset.rs:302
3872024-12-21T01:14:21.736ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_192cce10-69a8-4ed6-a459-9ca86a7206ec file = sled-storage/src/dataset.rs:388
3882024-12-21T01:14:21.744ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:440
3892024-12-21T01:14:21.744ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:357
3902024-12-21T01:14:21.744ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:342
3912024-12-21T01:14:21.744ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3922024-12-21T01:14:21.744ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } path = /scratch/u2_7.vdev
3932024-12-21T01:14:21.753ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_7.vdev file = sled-hardware/src/disk.rs:367
3942024-12-21T01:14:21.753ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 34789575-52b6-411b-9614-90f11629cb04 (zpool)
3952024-12-21T01:14:21.851ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }
3962024-12-21T01:14:21.860ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:239
3972024-12-21T01:14:21.860ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:241
3982024-12-21T01:14:21.860ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:245
3992024-12-21T01:14:21.860ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:247
4002024-12-21T01:14:21.860ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4012024-12-21T01:14:21.860ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4022024-12-21T01:14:21.860ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
4032024-12-21T01:14:21.901ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4042024-12-21T01:14:21.909ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_34789575-52b6-411b-9614-90f11629cb04/crypt/zone file = sled-storage/src/dataset.rs:302
4052024-12-21T01:14:22.050ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_34789575-52b6-411b-9614-90f11629cb04 file = sled-storage/src/dataset.rs:388
4062024-12-21T01:14:22.058ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:440
4072024-12-21T01:14:22.058ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:357
4082024-12-21T01:14:22.058ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:342
4092024-12-21T01:14:22.058ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4102024-12-21T01:14:22.058ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } path = /scratch/u2_8.vdev
4112024-12-21T01:14:22.068ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_8.vdev file = sled-hardware/src/disk.rs:367
4122024-12-21T01:14:22.068ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = e80b2509-2264-41c1-be0f-a3173a243a47 (zpool)
4132024-12-21T01:14:22.169ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }
4142024-12-21T01:14:22.177ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:239
4152024-12-21T01:14:22.178ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:241
4162024-12-21T01:14:22.178ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:245
4172024-12-21T01:14:22.178ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:247
4182024-12-21T01:14:22.178ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4192024-12-21T01:14:22.178ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4202024-12-21T01:14:22.178ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:259
4212024-12-21T01:14:22.214ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4222024-12-21T01:14:22.221ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_e80b2509-2264-41c1-be0f-a3173a243a47/crypt/zone file = sled-storage/src/dataset.rs:302
4232024-12-21T01:14:22.361ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/b1eebc13-191d-4a5d-a35e-84ae64fecac9) file = sled-agent/src/probe_manager.rs:231 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
4242024-12-21T01:14:22.388ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_e80b2509-2264-41c1-be0f-a3173a243a47 file = sled-storage/src/dataset.rs:388
4252024-12-21T01:14:22.397ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:440
4262024-12-21T01:14:22.397ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:357
4272024-12-21T01:14:22.397ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4282024-12-21T01:14:22.397ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(2), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 39f3a11f-ebbe-4984-ad03-60e64a112dfb (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 1bf7a526-12df-41e0-8b64-89b419ddc8ee (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
4292024-12-21T01:14:22.397ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
4302024-12-21T01:14:22.397ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
4312024-12-21T01:14:22.397ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4322024-12-21T01:14:22.397ZINFOSledAgent: physical disks ensure: Updated storage file = sled-agent/src/sled_agent.rs:897 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
4332024-12-21T01:14:22.397ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
4342024-12-21T01:14:22.397ZINFOSledAgent: physical disks ensure: Propagating new generation of disks file = sled-agent/src/sled_agent.rs:923 generation = Generation(2) sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
4352024-12-21T01:14:22.397ZINFOSledAgent: physical disks ensure: Updated storage monitor file = sled-agent/src/sled_agent.rs:928 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
4362024-12-21T01:14:22.398ZINFOSledAgent: physical disks ensure: Updated zone bundler file = sled-agent/src/sled_agent.rs:933 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
4372024-12-21T01:14:22.398ZINFOSledAgent (ProbeManager): use_only_these_disks: Processing new request file = sled-agent/src/probe_manager.rs:122 gen = Generation(2) sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
4382024-12-21T01:14:22.398ZINFOSledAgent: physical disks ensure: Updated probes file = sled-agent/src/sled_agent.rs:939 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
4392024-12-21T01:14:22.398ZINFOSledAgent (InstanceManager): use_only_these_disks: Processing new request file = sled-agent/src/instance_manager.rs:799 gen = Generation(2)
4402024-12-21T01:14:22.398ZINFOSledAgent: physical disks ensure: Updated instances file = sled-agent/src/sled_agent.rs:944 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
4412024-12-21T01:14:22.398ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3712533 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:55110 req_id = 23dafde3-4396-4685-bd4c-95946c75a10f response_code = 200 uri = /omicron-physical-disks
4422024-12-21T01:14:22.429ZINFOSledAgent (RSS): attempting to set up sled's datasets: DatasetsConfig { generation: Generation(1), datasets: {030a3a28-31ad-46b4-b5af-766222d875ff (dataset): DatasetConfig { id: 030a3a28-31ad-46b4-b5af-766222d875ff (dataset), name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0d432428-abbf-4fe3-93a5-c62bf9fc69ed (dataset): DatasetConfig { id: 0d432428-abbf-4fe3-93a5-c62bf9fc69ed (dataset), name: DatasetName { pool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0ebb33a5-bb24-4276-bef0-58efdea0d85e (dataset): DatasetConfig { id: 0ebb33a5-bb24-4276-bef0-58efdea0d85e (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 1396daa0-84e5-4cb4-90f2-0d53b2b74ecd (dataset): DatasetConfig { id: 1396daa0-84e5-4cb4-90f2-0d53b2b74ecd (dataset), name: DatasetName { pool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 17871109-43fc-4c81-807b-6838c79a66a7 (dataset): DatasetConfig { id: 17871109-43fc-4c81-807b-6838c79a66a7 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 196dbea2-776f-409b-8478-703206300ae0 (dataset): DatasetConfig { id: 196dbea2-776f-409b-8478-703206300ae0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1c88c5c9-ae84-487e-9f64-d93abab92279 (dataset): DatasetConfig { id: 1c88c5c9-ae84-487e-9f64-d93abab92279 (dataset), name: DatasetName { pool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 1d50ad0e-0ca3-4aa4-8eb3-c281efcf08a6 (dataset): DatasetConfig { id: 1d50ad0e-0ca3-4aa4-8eb3-c281efcf08a6 (dataset), name: DatasetName { pool_name: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2b5d794d-cdfe-4b61-bf96-bae1983a0a6d (dataset): DatasetConfig { id: 2b5d794d-cdfe-4b61-bf96-bae1983a0a6d (dataset), name: DatasetName { pool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2e827423-eb2b-49c2-b0d8-bc287cf36094 (dataset): DatasetConfig { id: 2e827423-eb2b-49c2-b0d8-bc287cf36094 (dataset), name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2ec8a602-ba70-4e43-ac00-60819b8373d2 (dataset): DatasetConfig { id: 2ec8a602-ba70-4e43-ac00-60819b8373d2 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2f7a12d8-d1e3-4d4b-be50-f1c597253275 (dataset): DatasetConfig { id: 2f7a12d8-d1e3-4d4b-be50-f1c597253275 (dataset), name: DatasetName { pool_name: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2fb82ca2-d0ef-4600-85cc-6bf06e72714a (dataset): DatasetConfig { id: 2fb82ca2-d0ef-4600-85cc-6bf06e72714a (dataset), name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2fc7e991-5854-4fb2-8a02-5fd4ab192171 (dataset): DatasetConfig { id: 2fc7e991-5854-4fb2-8a02-5fd4ab192171 (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 335bb8cc-2942-4c74-9ed5-8cf8432237bc (dataset): DatasetConfig { id: 335bb8cc-2942-4c74-9ed5-8cf8432237bc (dataset), name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 36686acb-d4f6-40a4-a63c-d4a950a08fd6 (dataset): DatasetConfig { id: 36686acb-d4f6-40a4-a63c-d4a950a08fd6 (dataset), name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 38a89ac8-a7dc-44b6-b20c-3cd75b78c8fb (dataset): DatasetConfig { id: 38a89ac8-a7dc-44b6-b20c-3cd75b78c8fb (dataset), name: DatasetName { pool_name: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3f6dcda7-d89d-4cad-907f-2f67cb5e006a (dataset): DatasetConfig { id: 3f6dcda7-d89d-4cad-907f-2f67cb5e006a (dataset), name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 3fca646f-5ec6-43e2-9c66-d5b266ae0592 (dataset): DatasetConfig { id: 3fca646f-5ec6-43e2-9c66-d5b266ae0592 (dataset), name: DatasetName { pool_name: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }, kind: TransientZone { name: "oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 40568b33-2a00-4c30-8b9b-2f4cd9e85266 (dataset): DatasetConfig { id: 40568b33-2a00-4c30-8b9b-2f4cd9e85266 (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 40d6199a-2a3c-4bd2-99e2-f0dcd2a76f21 (dataset): DatasetConfig { id: 40d6199a-2a3c-4bd2-99e2-f0dcd2a76f21 (dataset), name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 41b3606f-d95b-4a19-94d4-91ef9784eee0 (dataset): DatasetConfig { id: 41b3606f-d95b-4a19-94d4-91ef9784eee0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 47203f50-a5f6-4fbf-b190-236c77c64435 (dataset): DatasetConfig { id: 47203f50-a5f6-4fbf-b190-236c77c64435 (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 488bc733-e10b-4bff-86b0-a0087cfd765d (dataset): DatasetConfig { id: 488bc733-e10b-4bff-86b0-a0087cfd765d (dataset), name: DatasetName { pool_name: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 48bb89ec-63ef-4163-b900-dbff3d623874 (dataset): DatasetConfig { id: 48bb89ec-63ef-4163-b900-dbff3d623874 (dataset), name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4a561f4a-09b6-4ffe-b251-014032e748d7 (dataset): DatasetConfig { id: 4a561f4a-09b6-4ffe-b251-014032e748d7 (dataset), name: DatasetName { pool_name: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4e710363-9005-477e-82d2-2dfcf536c046 (dataset): DatasetConfig { id: 4e710363-9005-477e-82d2-2dfcf536c046 (dataset), name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 4eebaabc-fd84-4a95-a062-41e71d80163e (dataset): DatasetConfig { id: 4eebaabc-fd84-4a95-a062-41e71d80163e (dataset), name: DatasetName { pool_name: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 50305a65-4cc5-44d5-8e24-91126b5c5fa1 (dataset): DatasetConfig { id: 50305a65-4cc5-44d5-8e24-91126b5c5fa1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 511780d9-e72b-4edd-a478-880db1ead02d (dataset): DatasetConfig { id: 511780d9-e72b-4edd-a478-880db1ead02d (dataset), name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 565c2069-f390-4868-99b0-1f02b16fac3d (dataset): DatasetConfig { id: 565c2069-f390-4868-99b0-1f02b16fac3d (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 585444be-ddf4-4b42-b140-ba492c26a58d (dataset): DatasetConfig { id: 585444be-ddf4-4b42-b140-ba492c26a58d (dataset), name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5a365cd0-99f5-4192-8f69-6b954e2abed2 (dataset): DatasetConfig { id: 5a365cd0-99f5-4192-8f69-6b954e2abed2 (dataset), name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5cfd7658-0de5-4725-b241-a2c656387ebc (dataset): DatasetConfig { id: 5cfd7658-0de5-4725-b241-a2c656387ebc (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5f5e06f9-4163-4206-a9da-cc9f3a8b3a74 (dataset): DatasetConfig { id: 5f5e06f9-4163-4206-a9da-cc9f3a8b3a74 (dataset), name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6493b26c-1a8b-49aa-b870-e0b11b397d71 (dataset): DatasetConfig { id: 6493b26c-1a8b-49aa-b870-e0b11b397d71 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Clickhouse }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6555e7e1-ecc3-4542-86a9-a867498e331f (dataset): DatasetConfig { id: 6555e7e1-ecc3-4542-86a9-a867498e331f (dataset), name: DatasetName { pool_name: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 658090e9-19f0-4c76-8a55-d5ecb2d5e791 (dataset): DatasetConfig { id: 658090e9-19f0-4c76-8a55-d5ecb2d5e791 (dataset), name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 65f623a8-751a-4433-991a-8a0bbab0a8a1 (dataset): DatasetConfig { id: 65f623a8-751a-4433-991a-8a0bbab0a8a1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6774ad1a-fc17-4da0-a8b3-a30f5075312a (dataset): DatasetConfig { id: 6774ad1a-fc17-4da0-a8b3-a30f5075312a (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6ba9c380-69cf-4d72-8502-0b07bedda703 (dataset): DatasetConfig { id: 6ba9c380-69cf-4d72-8502-0b07bedda703 (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6bd5ca84-b3f7-4dbd-aaf7-9afeebdabedd (dataset): DatasetConfig { id: 6bd5ca84-b3f7-4dbd-aaf7-9afeebdabedd (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 7489f682-a26f-4580-a82a-f0592ed1b7be (dataset): DatasetConfig { id: 7489f682-a26f-4580-a82a-f0592ed1b7be (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 756f5d1e-9bce-46da-9f32-a759cc1089f5 (dataset): DatasetConfig { id: 756f5d1e-9bce-46da-9f32-a759cc1089f5 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 7858f796-886c-4a3f-ac02-bcbbd6032296 (dataset): DatasetConfig { id: 7858f796-886c-4a3f-ac02-bcbbd6032296 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 785aa456-8975-4d53-a89b-a6b7e7c71991 (dataset): DatasetConfig { id: 785aa456-8975-4d53-a89b-a6b7e7c71991 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 78b6f82f-b7f9-4eac-9dea-86ea88e0c072 (dataset): DatasetConfig { id: 78b6f82f-b7f9-4eac-9dea-86ea88e0c072 (dataset), name: DatasetName { pool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8141253a-1674-44de-8355-ea9ae53eeb94 (dataset): DatasetConfig { id: 8141253a-1674-44de-8355-ea9ae53eeb94 (dataset), name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 84d2ff9f-75df-4906-9c00-651d274c3a8e (dataset): DatasetConfig { id: 84d2ff9f-75df-4906-9c00-651d274c3a8e (dataset), name: DatasetName { pool_name: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 8657c64a-c961-44bd-b6a7-29c4a6a0a639 (dataset): DatasetConfig { id: 8657c64a-c961-44bd-b6a7-29c4a6a0a639 (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8c7f38a7-370e-43a7-9ed4-67c5f816d225 (dataset): DatasetConfig { id: 8c7f38a7-370e-43a7-9ed4-67c5f816d225 (dataset), name: DatasetName { pool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, kind: TransientZone { name: "oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 92983adb-fa54-4454-9558-945e04c6f687 (dataset): DatasetConfig { id: 92983adb-fa54-4454-9558-945e04c6f687 (dataset), name: DatasetName { pool_name: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a004e97e-00ad-410c-b222-c041b1f3c574 (dataset): DatasetConfig { id: a004e97e-00ad-410c-b222-c041b1f3c574 (dataset), name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a131d716-0ace-4a3d-bfaa-c0a4b87befca (dataset): DatasetConfig { id: a131d716-0ace-4a3d-bfaa-c0a4b87befca (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ac08167a-a592-419a-91c9-52466bc4430c (dataset): DatasetConfig { id: ac08167a-a592-419a-91c9-52466bc4430c (dataset), name: DatasetName { pool_name: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, bceae143-045c-407b-ac12-c6a0319ecaac (dataset): DatasetConfig { id: bceae143-045c-407b-ac12-c6a0319ecaac (dataset), name: DatasetName { pool_name: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c3c23efb-d9db-495b-bcb7-e4273e82e24d (dataset): DatasetConfig { id: c3c23efb-d9db-495b-bcb7-e4273e82e24d (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c4d3d8f8-c41f-4f59-8463-de73e86a8e32 (dataset): DatasetConfig { id: c4d3d8f8-c41f-4f59-8463-de73e86a8e32 (dataset), name: DatasetName { pool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ca4ae51c-a89a-444f-bb0f-e3db2cb19589 (dataset): DatasetConfig { id: ca4ae51c-a89a-444f-bb0f-e3db2cb19589 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ceed6215-1926-4662-92c6-c862ab43ce93 (dataset): DatasetConfig { id: ceed6215-1926-4662-92c6-c862ab43ce93 (dataset), name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d1431ed7-db7c-4408-8646-5ce125ff7769 (dataset): DatasetConfig { id: d1431ed7-db7c-4408-8646-5ce125ff7769 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: TransientZone { name: "oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d735d744-9997-4f11-aedf-0cf850d6531a (dataset): DatasetConfig { id: d735d744-9997-4f11-aedf-0cf850d6531a (dataset), name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, dd9fd42a-ef23-468f-94e1-63107cfaec8b (dataset): DatasetConfig { id: dd9fd42a-ef23-468f-94e1-63107cfaec8b (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, dddfa60a-fe22-4de6-88be-0f6b0820fb0e (dataset): DatasetConfig { id: dddfa60a-fe22-4de6-88be-0f6b0820fb0e (dataset), name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e02505d7-e5b7-45ff-b932-34ed76997583 (dataset): DatasetConfig { id: e02505d7-e5b7-45ff-b932-34ed76997583 (dataset), name: DatasetName { pool_name: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ffeb4a91-f5cd-48cf-ad72-4fd0b410c2cc (dataset): DatasetConfig { id: ffeb4a91-f5cd-48cf-ad72-4fd0b410c2cc (dataset), name: DatasetName { pool_name: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }} } file = sled-agent/src/rack_setup/service.rs:460 sled_address = [fd00:1122:3344:101::1]:12345
4432024-12-21T01:14:22.430ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:54175
4442024-12-21T01:14:22.430ZINFOSledAgent: datasets ensure file = sled-agent/src/sled_agent.rs:859 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
4452024-12-21T01:14:22.430ZINFOSledAgent (StorageManager): Received DatasetsEnsure { config: DatasetsConfig { generation: Generation(1), datasets: {030a3a28-31ad-46b4-b5af-766222d875ff (dataset): DatasetConfig { id: 030a3a28-31ad-46b4-b5af-766222d875ff (dataset), name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0d432428-abbf-4fe3-93a5-c62bf9fc69ed (dataset): DatasetConfig { id: 0d432428-abbf-4fe3-93a5-c62bf9fc69ed (dataset), name: DatasetName { pool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0ebb33a5-bb24-4276-bef0-58efdea0d85e (dataset): DatasetConfig { id: 0ebb33a5-bb24-4276-bef0-58efdea0d85e (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 1396daa0-84e5-4cb4-90f2-0d53b2b74ecd (dataset): DatasetConfig { id: 1396daa0-84e5-4cb4-90f2-0d53b2b74ecd (dataset), name: DatasetName { pool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 17871109-43fc-4c81-807b-6838c79a66a7 (dataset): DatasetConfig { id: 17871109-43fc-4c81-807b-6838c79a66a7 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 196dbea2-776f-409b-8478-703206300ae0 (dataset): DatasetConfig { id: 196dbea2-776f-409b-8478-703206300ae0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1c88c5c9-ae84-487e-9f64-d93abab92279 (dataset): DatasetConfig { id: 1c88c5c9-ae84-487e-9f64-d93abab92279 (dataset), name: DatasetName { pool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 1d50ad0e-0ca3-4aa4-8eb3-c281efcf08a6 (dataset): DatasetConfig { id: 1d50ad0e-0ca3-4aa4-8eb3-c281efcf08a6 (dataset), name: DatasetName { pool_name: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2b5d794d-cdfe-4b61-bf96-bae1983a0a6d (dataset): DatasetConfig { id: 2b5d794d-cdfe-4b61-bf96-bae1983a0a6d (dataset), name: DatasetName { pool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2e827423-eb2b-49c2-b0d8-bc287cf36094 (dataset): DatasetConfig { id: 2e827423-eb2b-49c2-b0d8-bc287cf36094 (dataset), name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2ec8a602-ba70-4e43-ac00-60819b8373d2 (dataset): DatasetConfig { id: 2ec8a602-ba70-4e43-ac00-60819b8373d2 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2f7a12d8-d1e3-4d4b-be50-f1c597253275 (dataset): DatasetConfig { id: 2f7a12d8-d1e3-4d4b-be50-f1c597253275 (dataset), name: DatasetName { pool_name: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2fb82ca2-d0ef-4600-85cc-6bf06e72714a (dataset): DatasetConfig { id: 2fb82ca2-d0ef-4600-85cc-6bf06e72714a (dataset), name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2fc7e991-5854-4fb2-8a02-5fd4ab192171 (dataset): DatasetConfig { id: 2fc7e991-5854-4fb2-8a02-5fd4ab192171 (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 335bb8cc-2942-4c74-9ed5-8cf8432237bc (dataset): DatasetConfig { id: 335bb8cc-2942-4c74-9ed5-8cf8432237bc (dataset), name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 36686acb-d4f6-40a4-a63c-d4a950a08fd6 (dataset): DatasetConfig { id: 36686acb-d4f6-40a4-a63c-d4a950a08fd6 (dataset), name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 38a89ac8-a7dc-44b6-b20c-3cd75b78c8fb (dataset): DatasetConfig { id: 38a89ac8-a7dc-44b6-b20c-3cd75b78c8fb (dataset), name: DatasetName { pool_name: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3f6dcda7-d89d-4cad-907f-2f67cb5e006a (dataset): DatasetConfig { id: 3f6dcda7-d89d-4cad-907f-2f67cb5e006a (dataset), name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 3fca646f-5ec6-43e2-9c66-d5b266ae0592 (dataset): DatasetConfig { id: 3fca646f-5ec6-43e2-9c66-d5b266ae0592 (dataset), name: DatasetName { pool_name: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }, kind: TransientZone { name: "oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 40568b33-2a00-4c30-8b9b-2f4cd9e85266 (dataset): DatasetConfig { id: 40568b33-2a00-4c30-8b9b-2f4cd9e85266 (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 40d6199a-2a3c-4bd2-99e2-f0dcd2a76f21 (dataset): DatasetConfig { id: 40d6199a-2a3c-4bd2-99e2-f0dcd2a76f21 (dataset), name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 41b3606f-d95b-4a19-94d4-91ef9784eee0 (dataset): DatasetConfig { id: 41b3606f-d95b-4a19-94d4-91ef9784eee0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 47203f50-a5f6-4fbf-b190-236c77c64435 (dataset): DatasetConfig { id: 47203f50-a5f6-4fbf-b190-236c77c64435 (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 488bc733-e10b-4bff-86b0-a0087cfd765d (dataset): DatasetConfig { id: 488bc733-e10b-4bff-86b0-a0087cfd765d (dataset), name: DatasetName { pool_name: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 48bb89ec-63ef-4163-b900-dbff3d623874 (dataset): DatasetConfig { id: 48bb89ec-63ef-4163-b900-dbff3d623874 (dataset), name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4a561f4a-09b6-4ffe-b251-014032e748d7 (dataset): DatasetConfig { id: 4a561f4a-09b6-4ffe-b251-014032e748d7 (dataset), name: DatasetName { pool_name: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4e710363-9005-477e-82d2-2dfcf536c046 (dataset): DatasetConfig { id: 4e710363-9005-477e-82d2-2dfcf536c046 (dataset), name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 4eebaabc-fd84-4a95-a062-41e71d80163e (dataset): DatasetConfig { id: 4eebaabc-fd84-4a95-a062-41e71d80163e (dataset), name: DatasetName { pool_name: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 50305a65-4cc5-44d5-8e24-91126b5c5fa1 (dataset): DatasetConfig { id: 50305a65-4cc5-44d5-8e24-91126b5c5fa1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 511780d9-e72b-4edd-a478-880db1ead02d (dataset): DatasetConfig { id: 511780d9-e72b-4edd-a478-880db1ead02d (dataset), name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 565c2069-f390-4868-99b0-1f02b16fac3d (dataset): DatasetConfig { id: 565c2069-f390-4868-99b0-1f02b16fac3d (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 585444be-ddf4-4b42-b140-ba492c26a58d (dataset): DatasetConfig { id: 585444be-ddf4-4b42-b140-ba492c26a58d (dataset), name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5a365cd0-99f5-4192-8f69-6b954e2abed2 (dataset): DatasetConfig { id: 5a365cd0-99f5-4192-8f69-6b954e2abed2 (dataset), name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5cfd7658-0de5-4725-b241-a2c656387ebc (dataset): DatasetConfig { id: 5cfd7658-0de5-4725-b241-a2c656387ebc (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5f5e06f9-4163-4206-a9da-cc9f3a8b3a74 (dataset): DatasetConfig { id: 5f5e06f9-4163-4206-a9da-cc9f3a8b3a74 (dataset), name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6493b26c-1a8b-49aa-b870-e0b11b397d71 (dataset): DatasetConfig { id: 6493b26c-1a8b-49aa-b870-e0b11b397d71 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Clickhouse }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6555e7e1-ecc3-4542-86a9-a867498e331f (dataset): DatasetConfig { id: 6555e7e1-ecc3-4542-86a9-a867498e331f (dataset), name: DatasetName { pool_name: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 658090e9-19f0-4c76-8a55-d5ecb2d5e791 (dataset): DatasetConfig { id: 658090e9-19f0-4c76-8a55-d5ecb2d5e791 (dataset), name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 65f623a8-751a-4433-991a-8a0bbab0a8a1 (dataset): DatasetConfig { id: 65f623a8-751a-4433-991a-8a0bbab0a8a1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6774ad1a-fc17-4da0-a8b3-a30f5075312a (dataset): DatasetConfig { id: 6774ad1a-fc17-4da0-a8b3-a30f5075312a (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6ba9c380-69cf-4d72-8502-0b07bedda703 (dataset): DatasetConfig { id: 6ba9c380-69cf-4d72-8502-0b07bedda703 (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 6bd5ca84-b3f7-4dbd-aaf7-9afeebdabedd (dataset): DatasetConfig { id: 6bd5ca84-b3f7-4dbd-aaf7-9afeebdabedd (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 7489f682-a26f-4580-a82a-f0592ed1b7be (dataset): DatasetConfig { id: 7489f682-a26f-4580-a82a-f0592ed1b7be (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 756f5d1e-9bce-46da-9f32-a759cc1089f5 (dataset): DatasetConfig { id: 756f5d1e-9bce-46da-9f32-a759cc1089f5 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 7858f796-886c-4a3f-ac02-bcbbd6032296 (dataset): DatasetConfig { id: 7858f796-886c-4a3f-ac02-bcbbd6032296 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 785aa456-8975-4d53-a89b-a6b7e7c71991 (dataset): DatasetConfig { id: 785aa456-8975-4d53-a89b-a6b7e7c71991 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 78b6f82f-b7f9-4eac-9dea-86ea88e0c072 (dataset): DatasetConfig { id: 78b6f82f-b7f9-4eac-9dea-86ea88e0c072 (dataset), name: DatasetName { pool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8141253a-1674-44de-8355-ea9ae53eeb94 (dataset): DatasetConfig { id: 8141253a-1674-44de-8355-ea9ae53eeb94 (dataset), name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 84d2ff9f-75df-4906-9c00-651d274c3a8e (dataset): DatasetConfig { id: 84d2ff9f-75df-4906-9c00-651d274c3a8e (dataset), name: DatasetName { pool_name: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 8657c64a-c961-44bd-b6a7-29c4a6a0a639 (dataset): DatasetConfig { id: 8657c64a-c961-44bd-b6a7-29c4a6a0a639 (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8c7f38a7-370e-43a7-9ed4-67c5f816d225 (dataset): DatasetConfig { id: 8c7f38a7-370e-43a7-9ed4-67c5f816d225 (dataset), name: DatasetName { pool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, kind: TransientZone { name: "oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 92983adb-fa54-4454-9558-945e04c6f687 (dataset): DatasetConfig { id: 92983adb-fa54-4454-9558-945e04c6f687 (dataset), name: DatasetName { pool_name: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a004e97e-00ad-410c-b222-c041b1f3c574 (dataset): DatasetConfig { id: a004e97e-00ad-410c-b222-c041b1f3c574 (dataset), name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, a131d716-0ace-4a3d-bfaa-c0a4b87befca (dataset): DatasetConfig { id: a131d716-0ace-4a3d-bfaa-c0a4b87befca (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ac08167a-a592-419a-91c9-52466bc4430c (dataset): DatasetConfig { id: ac08167a-a592-419a-91c9-52466bc4430c (dataset), name: DatasetName { pool_name: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, bceae143-045c-407b-ac12-c6a0319ecaac (dataset): DatasetConfig { id: bceae143-045c-407b-ac12-c6a0319ecaac (dataset), name: DatasetName { pool_name: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c3c23efb-d9db-495b-bcb7-e4273e82e24d (dataset): DatasetConfig { id: c3c23efb-d9db-495b-bcb7-e4273e82e24d (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, c4d3d8f8-c41f-4f59-8463-de73e86a8e32 (dataset): DatasetConfig { id: c4d3d8f8-c41f-4f59-8463-de73e86a8e32 (dataset), name: DatasetName { pool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ca4ae51c-a89a-444f-bb0f-e3db2cb19589 (dataset): DatasetConfig { id: ca4ae51c-a89a-444f-bb0f-e3db2cb19589 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ceed6215-1926-4662-92c6-c862ab43ce93 (dataset): DatasetConfig { id: ceed6215-1926-4662-92c6-c862ab43ce93 (dataset), name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d1431ed7-db7c-4408-8646-5ce125ff7769 (dataset): DatasetConfig { id: d1431ed7-db7c-4408-8646-5ce125ff7769 (dataset), name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: TransientZone { name: "oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d735d744-9997-4f11-aedf-0cf850d6531a (dataset): DatasetConfig { id: d735d744-9997-4f11-aedf-0cf850d6531a (dataset), name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, dd9fd42a-ef23-468f-94e1-63107cfaec8b (dataset): DatasetConfig { id: dd9fd42a-ef23-468f-94e1-63107cfaec8b (dataset), name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, dddfa60a-fe22-4de6-88be-0f6b0820fb0e (dataset): DatasetConfig { id: dddfa60a-fe22-4de6-88be-0f6b0820fb0e (dataset), name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e02505d7-e5b7-45ff-b932-34ed76997583 (dataset): DatasetConfig { id: e02505d7-e5b7-45ff-b932-34ed76997583 (dataset), name: DatasetName { pool_name: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ffeb4a91-f5cd-48cf-ad72-4fd0b410c2cc (dataset): DatasetConfig { id: ffeb4a91-f5cd-48cf-ad72-4fd0b410c2cc (dataset), name: DatasetName { pool_name: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }} }, tx: ... } file = sled-storage/src/manager.rs:607
4462024-12-21T01:14:22.430ZINFOSledAgent (StorageManager): No ledger in /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-datasets.json file = common/src/ledger.rs:188 request = datasets_ensure
4472024-12-21T01:14:22.430ZINFOSledAgent (StorageManager): No ledger in /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-datasets.json file = common/src/ledger.rs:188 request = datasets_ensure
4482024-12-21T01:14:22.430ZINFOSledAgent (StorageManager): No previously-stored 'requested datasets', creating new ledger file = sled-storage/src/manager.rs:909 request = datasets_ensure
4492024-12-21T01:14:22.446ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4502024-12-21T01:14:22.446ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4512024-12-21T01:14:22.446ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_7e02d0c4-4e5d-440d-94c1-593a67af3118/crucible" } file = sled-storage/src/manager.rs:1431
4522024-12-21T01:14:22.496ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4532024-12-21T01:14:22.496ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4542024-12-21T01:14:22.496ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/34789575-52b6-411b-9614-90f11629cb04/crypt/zone/oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a"), full_name: "oxp_34789575-52b6-411b-9614-90f11629cb04/crypt/zone/oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a" } file = sled-storage/src/manager.rs:1431
4552024-12-21T01:14:22.555ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4562024-12-21T01:14:22.555ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
4572024-12-21T01:14:22.555ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/debug"), full_name: "oxp_41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/debug" } file = sled-storage/src/manager.rs:1431
4582024-12-21T01:14:22.584ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4592024-12-21T01:14:22.584ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
4602024-12-21T01:14:22.584ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/34789575-52b6-411b-9614-90f11629cb04/crypt/zone"), full_name: "oxp_34789575-52b6-411b-9614-90f11629cb04/crypt/zone" } file = sled-storage/src/manager.rs:1431
4612024-12-21T01:14:22.613ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4622024-12-21T01:14:22.613ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
4632024-12-21T01:14:22.613ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/debug"), full_name: "oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/debug" } file = sled-storage/src/manager.rs:1431
4642024-12-21T01:14:22.642ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4652024-12-21T01:14:22.642ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4662024-12-21T01:14:22.642ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1431
4672024-12-21T01:14:22.692ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4682024-12-21T01:14:22.692ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
4692024-12-21T01:14:22.692ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/34789575-52b6-411b-9614-90f11629cb04/crypt/debug"), full_name: "oxp_34789575-52b6-411b-9614-90f11629cb04/crypt/debug" } file = sled-storage/src/manager.rs:1431
4702024-12-21T01:14:22.722ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4712024-12-21T01:14:22.722ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
4722024-12-21T01:14:22.722ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/e80b2509-2264-41c1-be0f-a3173a243a47/crypt/zone"), full_name: "oxp_e80b2509-2264-41c1-be0f-a3173a243a47/crypt/zone" } file = sled-storage/src/manager.rs:1431
4732024-12-21T01:14:22.751ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4742024-12-21T01:14:22.751ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4752024-12-21T01:14:22.751ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_34789575-52b6-411b-9614-90f11629cb04/crucible" } file = sled-storage/src/manager.rs:1431
4762024-12-21T01:14:22.800ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4772024-12-21T01:14:22.800ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4782024-12-21T01:14:22.800ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/7e02d0c4-4e5d-440d-94c1-593a67af3118/crypt/zone/oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b"), full_name: "oxp_7e02d0c4-4e5d-440d-94c1-593a67af3118/crypt/zone/oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b" } file = sled-storage/src/manager.rs:1431
4792024-12-21T01:14:22.860ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4802024-12-21T01:14:22.860ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4812024-12-21T01:14:22.860ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/external_dns" } file = sled-storage/src/manager.rs:1431
4822024-12-21T01:14:22.913ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4832024-12-21T01:14:22.913ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4842024-12-21T01:14:22.913ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739/crypt/zone/oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d"), full_name: "oxp_2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739/crypt/zone/oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d" } file = sled-storage/src/manager.rs:1431
4852024-12-21T01:14:22.974ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4862024-12-21T01:14:22.974ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4872024-12-21T01:14:22.974ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/22c20cb3-3659-48b7-b12d-b9ae21224e50/crypt/zone/oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21"), full_name: "oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crypt/zone/oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21" } file = sled-storage/src/manager.rs:1431
4882024-12-21T01:14:23.035ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4892024-12-21T01:14:23.035ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4902024-12-21T01:14:23.035ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/zone/oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1"), full_name: "oxp_41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/zone/oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1" } file = sled-storage/src/manager.rs:1431
4912024-12-21T01:14:23.094ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4922024-12-21T01:14:23.094ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
4932024-12-21T01:14:23.094ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/10325d85-ce1b-49aa-8039-7e9482ec8803/crypt/zone"), full_name: "oxp_10325d85-ce1b-49aa-8039-7e9482ec8803/crypt/zone" } file = sled-storage/src/manager.rs:1431
4942024-12-21T01:14:23.124ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4952024-12-21T01:14:23.124ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
4962024-12-21T01:14:23.124ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/10325d85-ce1b-49aa-8039-7e9482ec8803/crypt/debug"), full_name: "oxp_10325d85-ce1b-49aa-8039-7e9482ec8803/crypt/debug" } file = sled-storage/src/manager.rs:1431
4972024-12-21T01:14:23.153ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
4982024-12-21T01:14:23.153ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
4992024-12-21T01:14:23.153ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739/crucible" } file = sled-storage/src/manager.rs:1431
5002024-12-21T01:14:23.200ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5012024-12-21T01:14:23.200ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5022024-12-21T01:14:23.200ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/22c20cb3-3659-48b7-b12d-b9ae21224e50/crypt/debug"), full_name: "oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crypt/debug" } file = sled-storage/src/manager.rs:1431
5032024-12-21T01:14:23.229ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5042024-12-21T01:14:23.229ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5052024-12-21T01:14:23.229ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/192cce10-69a8-4ed6-a459-9ca86a7206ec/crypt/zone/oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732"), full_name: "oxp_192cce10-69a8-4ed6-a459-9ca86a7206ec/crypt/zone/oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732" } file = sled-storage/src/manager.rs:1431
5062024-12-21T01:14:23.290ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5072024-12-21T01:14:23.290ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5082024-12-21T01:14:23.290ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/internal_dns" } file = sled-storage/src/manager.rs:1431
5092024-12-21T01:14:23.340ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5102024-12-21T01:14:23.340ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5112024-12-21T01:14:23.340ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/22c20cb3-3659-48b7-b12d-b9ae21224e50/crypt/zone"), full_name: "oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crypt/zone" } file = sled-storage/src/manager.rs:1431
5122024-12-21T01:14:23.363ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/b1eebc13-191d-4a5d-a35e-84ae64fecac9) file = sled-agent/src/probe_manager.rs:231 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
5132024-12-21T01:14:23.370ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5142024-12-21T01:14:23.370ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5152024-12-21T01:14:23.370ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/internal_dns" } file = sled-storage/src/manager.rs:1431
5162024-12-21T01:14:23.421ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5172024-12-21T01:14:23.421ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5182024-12-21T01:14:23.421ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/zone/oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153"), full_name: "oxp_41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/zone/oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153" } file = sled-storage/src/manager.rs:1431
5192024-12-21T01:14:23.482ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5202024-12-21T01:14:23.482ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5212024-12-21T01:14:23.482ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/e80b2509-2264-41c1-be0f-a3173a243a47/crypt/zone/oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1"), full_name: "oxp_e80b2509-2264-41c1-be0f-a3173a243a47/crypt/zone/oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1" } file = sled-storage/src/manager.rs:1431
5222024-12-21T01:14:23.544ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5232024-12-21T01:14:23.544ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5242024-12-21T01:14:23.544ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/7e02d0c4-4e5d-440d-94c1-593a67af3118/crypt/zone/oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f"), full_name: "oxp_7e02d0c4-4e5d-440d-94c1-593a67af3118/crypt/zone/oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f" } file = sled-storage/src/manager.rs:1431
5252024-12-21T01:14:23.606ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5262024-12-21T01:14:23.606ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5272024-12-21T01:14:23.606ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/192cce10-69a8-4ed6-a459-9ca86a7206ec/crypt/zone"), full_name: "oxp_192cce10-69a8-4ed6-a459-9ca86a7206ec/crypt/zone" } file = sled-storage/src/manager.rs:1431
5282024-12-21T01:14:23.636ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5292024-12-21T01:14:23.636ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5302024-12-21T01:14:23.636ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/7e02d0c4-4e5d-440d-94c1-593a67af3118/crypt/debug"), full_name: "oxp_7e02d0c4-4e5d-440d-94c1-593a67af3118/crypt/debug" } file = sled-storage/src/manager.rs:1431
5312024-12-21T01:14:23.666ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5322024-12-21T01:14:23.666ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5332024-12-21T01:14:23.666ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739/crypt/debug"), full_name: "oxp_2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739/crypt/debug" } file = sled-storage/src/manager.rs:1431
5342024-12-21T01:14:23.696ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5352024-12-21T01:14:23.696ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5362024-12-21T01:14:23.696ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_192cce10-69a8-4ed6-a459-9ca86a7206ec/crucible" } file = sled-storage/src/manager.rs:1431
5372024-12-21T01:14:23.745ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5382024-12-21T01:14:23.745ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5392024-12-21T01:14:23.745ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crucible" } file = sled-storage/src/manager.rs:1431
5402024-12-21T01:14:23.793ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5412024-12-21T01:14:23.793ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5422024-12-21T01:14:23.793ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/zone/oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842"), full_name: "oxp_41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/zone/oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842" } file = sled-storage/src/manager.rs:1431
5432024-12-21T01:14:23.854ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5442024-12-21T01:14:23.854ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5452024-12-21T01:14:23.854ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/7e02d0c4-4e5d-440d-94c1-593a67af3118/crypt/zone"), full_name: "oxp_7e02d0c4-4e5d-440d-94c1-593a67af3118/crypt/zone" } file = sled-storage/src/manager.rs:1431
5462024-12-21T01:14:23.884ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5472024-12-21T01:14:23.884ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5482024-12-21T01:14:23.884ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/10325d85-ce1b-49aa-8039-7e9482ec8803/crypt/zone/oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261"), full_name: "oxp_10325d85-ce1b-49aa-8039-7e9482ec8803/crypt/zone/oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261" } file = sled-storage/src/manager.rs:1431
5492024-12-21T01:14:23.944ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5502024-12-21T01:14:23.944ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5512024-12-21T01:14:23.944ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/zone/oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0"), full_name: "oxp_41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/zone/oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0" } file = sled-storage/src/manager.rs:1431
5522024-12-21T01:14:24.003ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5532024-12-21T01:14:24.003ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5542024-12-21T01:14:24.003ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_7e02d0c4-4e5d-440d-94c1-593a67af3118/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1431
5552024-12-21T01:14:24.056ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5562024-12-21T01:14:24.056ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5572024-12-21T01:14:24.056ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/clickhouse" } file = sled-storage/src/manager.rs:1431
5582024-12-21T01:14:24.108ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5592024-12-21T01:14:24.108ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5602024-12-21T01:14:24.108ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/192cce10-69a8-4ed6-a459-9ca86a7206ec/crypt/debug"), full_name: "oxp_192cce10-69a8-4ed6-a459-9ca86a7206ec/crypt/debug" } file = sled-storage/src/manager.rs:1431
5612024-12-21T01:14:24.139ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5622024-12-21T01:14:24.140ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5632024-12-21T01:14:24.140ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_7e02d0c4-4e5d-440d-94c1-593a67af3118/crypt/internal_dns" } file = sled-storage/src/manager.rs:1431
5642024-12-21T01:14:24.189ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5652024-12-21T01:14:24.189ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5662024-12-21T01:14:24.189ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739/crypt/zone"), full_name: "oxp_2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739/crypt/zone" } file = sled-storage/src/manager.rs:1431
5672024-12-21T01:14:24.218ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5682024-12-21T01:14:24.218ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5692024-12-21T01:14:24.218ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/zone/oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5"), full_name: "oxp_41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/zone/oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5" } file = sled-storage/src/manager.rs:1431
5702024-12-21T01:14:24.279ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5712024-12-21T01:14:24.279ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5722024-12-21T01:14:24.279ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/external_dns" } file = sled-storage/src/manager.rs:1431
5732024-12-21T01:14:24.330ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5742024-12-21T01:14:24.330ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5752024-12-21T01:14:24.330ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/zone/oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719"), full_name: "oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/zone/oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719" } file = sled-storage/src/manager.rs:1431
5762024-12-21T01:14:24.365ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/b1eebc13-191d-4a5d-a35e-84ae64fecac9) file = sled-agent/src/probe_manager.rs:231 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
5772024-12-21T01:14:24.390ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5782024-12-21T01:14:24.390ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5792024-12-21T01:14:24.390ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1431
5802024-12-21T01:14:24.439ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5812024-12-21T01:14:24.440ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5822024-12-21T01:14:24.440ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/zone/oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf"), full_name: "oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/zone/oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf" } file = sled-storage/src/manager.rs:1431
5832024-12-21T01:14:24.498ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5842024-12-21T01:14:24.498ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5852024-12-21T01:14:24.498ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/zone/oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7"), full_name: "oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/zone/oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7" } file = sled-storage/src/manager.rs:1431
5862024-12-21T01:14:24.557ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5872024-12-21T01:14:24.557ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5882024-12-21T01:14:24.557ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crucible" } file = sled-storage/src/manager.rs:1431
5892024-12-21T01:14:24.605ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5902024-12-21T01:14:24.605ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5912024-12-21T01:14:24.605ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/34789575-52b6-411b-9614-90f11629cb04/crypt/zone/oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a"), full_name: "oxp_34789575-52b6-411b-9614-90f11629cb04/crypt/zone/oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a" } file = sled-storage/src/manager.rs:1431
5922024-12-21T01:14:24.665ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5932024-12-21T01:14:24.665ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
5942024-12-21T01:14:24.665ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_10325d85-ce1b-49aa-8039-7e9482ec8803/crucible" } file = sled-storage/src/manager.rs:1431
5952024-12-21T01:14:24.713ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5962024-12-21T01:14:24.713ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
5972024-12-21T01:14:24.713ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/e80b2509-2264-41c1-be0f-a3173a243a47/crypt/debug"), full_name: "oxp_e80b2509-2264-41c1-be0f-a3173a243a47/crypt/debug" } file = sled-storage/src/manager.rs:1431
5982024-12-21T01:14:24.743ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
5992024-12-21T01:14:24.743ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6002024-12-21T01:14:24.743ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1431
6012024-12-21T01:14:24.793ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6022024-12-21T01:14:24.793ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6032024-12-21T01:14:24.793ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/34789575-52b6-411b-9614-90f11629cb04/crypt/zone/oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d"), full_name: "oxp_34789575-52b6-411b-9614-90f11629cb04/crypt/zone/oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d" } file = sled-storage/src/manager.rs:1431
6042024-12-21T01:14:24.852ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6052024-12-21T01:14:24.853ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6062024-12-21T01:14:24.853ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739/crypt/zone/oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad"), full_name: "oxp_2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739/crypt/zone/oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad" } file = sled-storage/src/manager.rs:1431
6072024-12-21T01:14:24.912ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6082024-12-21T01:14:24.912ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6092024-12-21T01:14:24.912ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_10325d85-ce1b-49aa-8039-7e9482ec8803/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1431
6102024-12-21T01:14:24.963ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6112024-12-21T01:14:24.963ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
6122024-12-21T01:14:24.963ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/zone"), full_name: "oxp_41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/zone" } file = sled-storage/src/manager.rs:1431
6132024-12-21T01:14:24.992ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6142024-12-21T01:14:24.992ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6152024-12-21T01:14:24.992ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/192cce10-69a8-4ed6-a459-9ca86a7206ec/crypt/zone/oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4"), full_name: "oxp_192cce10-69a8-4ed6-a459-9ca86a7206ec/crypt/zone/oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4" } file = sled-storage/src/manager.rs:1431
6162024-12-21T01:14:25.049ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6172024-12-21T01:14:25.049ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6182024-12-21T01:14:25.049ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_e80b2509-2264-41c1-be0f-a3173a243a47/crucible" } file = sled-storage/src/manager.rs:1431
6192024-12-21T01:14:25.096ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6202024-12-21T01:14:25.096ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6212024-12-21T01:14:25.096ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/zone/oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490"), full_name: "oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/zone/oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490" } file = sled-storage/src/manager.rs:1431
6222024-12-21T01:14:25.154ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6232024-12-21T01:14:25.154ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6242024-12-21T01:14:25.154ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/34789575-52b6-411b-9614-90f11629cb04/crypt/zone/oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc"), full_name: "oxp_34789575-52b6-411b-9614-90f11629cb04/crypt/zone/oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc" } file = sled-storage/src/manager.rs:1431
6252024-12-21T01:14:25.213ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6262024-12-21T01:14:25.213ZINFOSledAgent (StorageManager): Old properties missing UUID file = sled-storage/src/manager.rs:998 request = datasets_ensure
6272024-12-21T01:14:25.213ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/zone"), full_name: "oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/zone" } file = sled-storage/src/manager.rs:1431
6282024-12-21T01:14:25.242ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6292024-12-21T01:14:25.242ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6302024-12-21T01:14:25.242ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/10325d85-ce1b-49aa-8039-7e9482ec8803/crypt/zone/oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39"), full_name: "oxp_10325d85-ce1b-49aa-8039-7e9482ec8803/crypt/zone/oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39" } file = sled-storage/src/manager.rs:1431
6312024-12-21T01:14:25.302ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6322024-12-21T01:14:25.302ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6332024-12-21T01:14:25.302ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/zone/oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906"), full_name: "oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/zone/oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906" } file = sled-storage/src/manager.rs:1431
6342024-12-21T01:14:25.364ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6352024-12-21T01:14:25.364ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6362024-12-21T01:14:25.364ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/22c20cb3-3659-48b7-b12d-b9ae21224e50/crypt/zone/oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36"), full_name: "oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crypt/zone/oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36" } file = sled-storage/src/manager.rs:1431
6372024-12-21T01:14:25.368ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/b1eebc13-191d-4a5d-a35e-84ae64fecac9) file = sled-agent/src/probe_manager.rs:231 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
6382024-12-21T01:14:25.426ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6392024-12-21T01:14:25.426ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6402024-12-21T01:14:25.426ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_41cc83ed-adbe-492f-ae31-7074b31692b0/crucible" } file = sled-storage/src/manager.rs:1431
6412024-12-21T01:14:25.474ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6422024-12-21T01:14:25.474ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6432024-12-21T01:14:25.474ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/7e02d0c4-4e5d-440d-94c1-593a67af3118/crypt/zone/oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4"), full_name: "oxp_7e02d0c4-4e5d-440d-94c1-593a67af3118/crypt/zone/oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4" } file = sled-storage/src/manager.rs:1431
6442024-12-21T01:14:25.534ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6452024-12-21T01:14:25.534ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6462024-12-21T01:14:25.534ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/e80b2509-2264-41c1-be0f-a3173a243a47/crypt/zone/oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4"), full_name: "oxp_e80b2509-2264-41c1-be0f-a3173a243a47/crypt/zone/oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4" } file = sled-storage/src/manager.rs:1431
6472024-12-21T01:14:25.595ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:1039 request = datasets_ensure
6482024-12-21T01:14:25.595ZINFOSledAgent (StorageManager): This dataset did not exist file = sled-storage/src/manager.rs:993 request = datasets_ensure
6492024-12-21T01:14:25.595ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/e80b2509-2264-41c1-be0f-a3173a243a47/crypt/zone/oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371"), full_name: "oxp_e80b2509-2264-41c1-be0f-a3173a243a47/crypt/zone/oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371" } file = sled-storage/src/manager.rs:1431
6502024-12-21T01:14:25.655ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/.omicron-datasets.json.tmp file = common/src/ledger.rs:199 request = datasets_ensure
6512024-12-21T01:14:25.655ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/.omicron-datasets.json.tmp file = common/src/ledger.rs:199 request = datasets_ensure
6522024-12-21T01:14:25.656ZINFOSledAgent: datasets ensure: Updated storage file = sled-agent/src/sled_agent.rs:861 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
6532024-12-21T01:14:25.656ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3226242 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:54175 req_id = 8188be9f-03f9-4007-ab71-8d0d22acd405 response_code = 200 uri = /datasets
6542024-12-21T01:14:25.688ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: d757960c-713d-487d-afb1-254f04de69f7 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 8e844631-8283-4702-ac45-290bbf4afea5 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 76b38c45-7399-47de-bb68-255b313cfd9f (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
6552024-12-21T01:14:25.688ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:59149
6562024-12-21T01:14:25.688ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
6572024-12-21T01:14:25.688ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }
6582024-12-21T01:14:25.698ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
6592024-12-21T01:14:25.698ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6602024-12-21T01:14:25.707ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
6612024-12-21T01:14:25.707ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6622024-12-21T01:14:25.716ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
6632024-12-21T01:14:25.716ZINFOSledAgent (ServiceManager): No ledger in /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-zones.json file = common/src/ledger.rs:188
6642024-12-21T01:14:25.716ZINFOSledAgent (ServiceManager): No ledger in /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-zones.json file = common/src/ledger.rs:188
6652024-12-21T01:14:25.716ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
6662024-12-21T01:14:25.759ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
6672024-12-21T01:14:25.759ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
6682024-12-21T01:14:25.759ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
6692024-12-21T01:14:25.812ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5 file = illumos-utils/src/zone.rs:325
6702024-12-21T01:14:25.816ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f file = illumos-utils/src/zone.rs:325
6712024-12-21T01:14:25.819ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7 file = illumos-utils/src/zone.rs:325
6722024-12-21T01:14:25.831ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5 file = illumos-utils/src/zone.rs:363
6732024-12-21T01:14:25.836ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f file = illumos-utils/src/zone.rs:363
6742024-12-21T01:14:25.840ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7 file = illumos-utils/src/zone.rs:363
6752024-12-21T01:14:26.370ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/b1eebc13-191d-4a5d-a35e-84ae64fecac9) file = sled-agent/src/probe_manager.rs:231 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
6762024-12-21T01:14:27.397ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService1'/> <propval type="astring" name="gateway" value='fd00:1122:3344:2::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:2::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:2::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:2::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6772024-12-21T01:14:27.397ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:2::, width: 64 }
6782024-12-21T01:14:27.397ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5
6792024-12-21T01:14:27.884ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService2'/> <propval type="astring" name="gateway" value='fd00:1122:3344:3::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:3::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:3::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:3::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6802024-12-21T01:14:28.133ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 1912036175591875858 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
6812024-12-21T01:14:28.133ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 576692204050020974 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
6822024-12-21T01:14:28.363ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService3'/> <propval type="astring" name="gateway" value='fd00:1122:3344:1::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:1::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:1::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:1::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6832024-12-21T01:14:28.363ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:3::, width: 64 }
6842024-12-21T01:14:28.363ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:1::, width: 64 }
6852024-12-21T01:14:28.371ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5") failed: Property not found. retry in 34.253227ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5
6862024-12-21T01:14:28.371ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f
6872024-12-21T01:14:28.377ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7
6882024-12-21T01:14:28.402ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists - OK file = sled-agent/src/services.rs:4307
6892024-12-21T01:14:28.402ZINFOSledAgent (ServiceManager): Ensuring there is a default route file = sled-agent/src/services.rs:4317 gateway = fd00:1122:3344:101::1
6902024-12-21T01:14:28.410ZINFOSledAgent (ServiceManager): Default route already exists file = sled-agent/src/services.rs:4331 gateway = fd00:1122:3344:101::1
6912024-12-21T01:14:28.410ZINFOSledAgent (ServiceManager): configuring dendrite service file = sled-agent/src/services.rs:4396
6922024-12-21T01:14:28.416ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5") failed: Property not found. retry in 132.409489ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5
6932024-12-21T01:14:28.511ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f") failed: Property not found. retry in 46.826904ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f
6942024-12-21T01:14:28.519ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7") failed: Property not found. retry in 64.045971ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7
6952024-12-21T01:14:28.557ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5") failed: Property not found. retry in 168.754682ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5
6962024-12-21T01:14:28.564ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f") failed: Property not found. retry in 72.286037ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f
6972024-12-21T01:14:28.634ZINFOSledAgent (ServiceManager): refreshed dendrite service with new configuration file = sled-agent/src/services.rs:4462
6982024-12-21T01:14:28.634ZINFOSledAgent (ServiceManager): configuring lldp service file = sled-agent/src/services.rs:4489
6992024-12-21T01:14:28.647ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7") failed: Property not found. retry in 65.121137ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7
7002024-12-21T01:14:28.675ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f") failed: Property not found. retry in 100.439007ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f
7012024-12-21T01:14:28.738ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7") failed: Property not found. retry in 254.843077ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7
7022024-12-21T01:14:28.750ZINFOSledAgent (ServiceManager): refreshed lldpd service with new configuration file = sled-agent/src/services.rs:4502
7032024-12-21T01:14:28.750ZINFOSledAgent (ServiceManager): configuring MGS service file = sled-agent/src/services.rs:4348
7042024-12-21T01:14:28.757ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5") failed: Property not found. retry in 534.935722ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5
7052024-12-21T01:14:28.791ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f") failed: Property not found. retry in 429.889339ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f
7062024-12-21T01:14:28.848ZINFOSledAgent (ServiceManager): refreshed MGS service with new configuration file = sled-agent/src/services.rs:4390
7072024-12-21T01:14:28.848ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:4469 rack_subnet = fd00:1122:3344::
7082024-12-21T01:14:28.883ZINFOSledAgent (ServiceManager): refreshed wicketd service with new configuration file = sled-agent/src/services.rs:4480
7092024-12-21T01:14:28.883ZINFOSledAgent (ServiceManager): configuring mgd service file = sled-agent/src/services.rs:4522
7102024-12-21T01:14:29.024ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7") failed: Property not found. retry in 355.515913ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7
7112024-12-21T01:14:29.064ZINFOSledAgent (ServiceManager): refreshed mgd service with new configuration file = sled-agent/src/services.rs:4554
7122024-12-21T01:14:29.064ZINFOSledAgent (ServiceManager): configuring mg-ddm service file = sled-agent/src/services.rs:4560
7132024-12-21T01:14:29.172ZINFOSledAgent (ServiceManager): refreshed mg-ddm service with new configuration file = sled-agent/src/services.rs:4601
7142024-12-21T01:14:29.172ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:370
7152024-12-21T01:14:29.205ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:420
7162024-12-21T01:14:29.232ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f") failed: Property not found. retry in 1.073722505s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f
7172024-12-21T01:14:29.242ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:717
7182024-12-21T01:14:29.242ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:762
7192024-12-21T01:14:29.243ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:765 uptime = 1
7202024-12-21T01:14:29.243ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.2.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: Some(\n None,\n ),\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n tx_eq: None,\n },\n },\n },\n} file = sled-agent/src/bootstrap/early_networking.rs:444
7212024-12-21T01:14:29.280ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
7222024-12-21T01:14:29.302ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5") failed: Property not found. retry in 636.61521ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5
7232024-12-21T01:14:29.332ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:607
7242024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
7252024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
7262024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
7272024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
7282024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
7292024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
7302024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
7312024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
7322024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
7332024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
7342024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
7352024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
7362024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
7372024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
7382024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
7392024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
7402024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
7412024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
7422024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
7432024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
7442024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
7452024-12-21T01:14:29.332ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
7462024-12-21T01:14:29.388ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7") failed: Property not found. retry in 431.432287ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7
7472024-12-21T01:14:29.501ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 11062531147977962647 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
7482024-12-21T01:14:29.502ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 14223862254712097596 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
7492024-12-21T01:14:29.830ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7") failed: Property not found. retry in 1.069543903s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7
7502024-12-21T01:14:29.949ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5") failed: Property not found. retry in 967.656952ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5
7512024-12-21T01:14:30.318ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f") failed: Property not found. retry in 1.088360653s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f
7522024-12-21T01:14:30.909ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7") failed: Property not found. retry in 1.135425797s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7
7532024-12-21T01:14:30.925ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5") failed: Property not found. retry in 1.211614628s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5
7542024-12-21T01:14:31.415ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f") failed: Property not found. retry in 885.409176ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f
7552024-12-21T01:14:32.053ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7") failed: Property not found. retry in 515.050015ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7
7562024-12-21T01:14:32.145ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5") failed: Property not found. retry in 1.199556885s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5
7572024-12-21T01:14:32.310ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f") failed: Property not found. retry in 873.919152ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f
7582024-12-21T01:14:32.578ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7") failed: Property not found. retry in 1.422025686s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7
7592024-12-21T01:14:33.193ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f") failed: Property not found. retry in 712.535498ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f
7602024-12-21T01:14:34.019ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_internal_dns_8e844631-8283-4702-ac45-290bbf4afea5
7612024-12-21T01:14:34.019ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_internal_dns_76b38c45-7399-47de-bb68-255b313cfd9f
7622024-12-21T01:14:34.019ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_internal_dns_d757960c-713d-487d-afb1-254f04de69f7
7632024-12-21T01:14:34.019ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7642024-12-21T01:14:34.020ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7652024-12-21T01:14:34.020ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8331731 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:59149 req_id = 566ef66d-02f5-44b8-9b73-854539807a7e response_code = 204 uri = /omicron-zones
7662024-12-21T01:14:34.022ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:658
7672024-12-21T01:14:34.295ZWARNSledAgent (RSS): failed to write DNS configuration (will retry in 114.124505ms) dns_config_addr = [fd00:1122:3344:1::1]:5353 error_message = Communication Error: error sending request for url (http://[fd00:1122:3344:1::1]:5353/config) file = sled-agent/src/rack_setup/service.rs:678
7682024-12-21T01:14:34.427ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:658
7692024-12-21T01:14:34.482ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:658
7702024-12-21T01:14:34.528ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:694
7712024-12-21T01:14:34.528ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
7722024-12-21T01:14:34.560ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
7732024-12-21T01:14:34.561ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
7742024-12-21T01:14:34.561ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
7752024-12-21T01:14:34.590ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: d757960c-713d-487d-afb1-254f04de69f7 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 8e844631-8283-4702-ac45-290bbf4afea5 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 76b38c45-7399-47de-bb68-255b313cfd9f (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: f1bcff64-b896-4076-8a2b-587d701f235d (service), filesystem_pool: Some(ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: edefca09-f432-47bc-95dc-35286f50fcc4, kind: Service { id: f1bcff64-b896-4076-8a2b-587d701f235d }, name: Name("ntp-f1bcff64-b896-4076-8a2b-587d701f235d"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
7762024-12-21T01:14:34.591ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:63465
7772024-12-21T01:14:34.591ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
7782024-12-21T01:14:34.591ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }
7792024-12-21T01:14:34.602ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
7802024-12-21T01:14:34.602ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7812024-12-21T01:14:34.612ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
7822024-12-21T01:14:34.612ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7832024-12-21T01:14:34.622ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
7842024-12-21T01:14:34.622ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-zones.json file = common/src/ledger.rs:177
7852024-12-21T01:14:34.622ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-zones.json file = common/src/ledger.rs:177
7862024-12-21T01:14:34.622ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
7872024-12-21T01:14:34.634ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
7882024-12-21T01:14:34.666ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
7892024-12-21T01:14:34.668ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
7902024-12-21T01:14:34.668ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
7912024-12-21T01:14:34.698ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte0", ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 134, 39]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 } }) }
7922024-12-21T01:14:34.698ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = boundary_ntp
7932024-12-21T01:14:34.698ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:83
7942024-12-21T01:14:34.698ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:90
7952024-12-21T01:14:34.698ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:118
7962024-12-21T01:14:34.699ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
7972024-12-21T01:14:34.722ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d file = illumos-utils/src/zone.rs:325
7982024-12-21T01:14:34.741ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d file = illumos-utils/src/zone.rs:363
7992024-12-21T01:14:35.390ZINFOSledAgent (ServiceManager): Profile for oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService4'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/chrony-setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="allow" value='fd00:1122:3344:100::/56'/> <propval type="boolean" name="boundary" value='true'/> <propval type="astring" name="boundary_pool" value='boundary-ntp.control-plane.oxide.internal'/> <propval type="astring" name="server" value='0.pool.ntp.org'/> </property_group> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="1.1.1.1" /> <value_node value="9.9.9.9" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/ntp"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte0'/> <propval type="astring" name="gateway" value='172.30.3.1'/> <propval type="astring" name="ip" value='172.30.3.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
8002024-12-21T01:14:35.390ZINFOSledAgent (ServiceManager): Booting oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d
8012024-12-21T01:14:35.537ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d") failed: Property not found. retry in 28.857596ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d
8022024-12-21T01:14:35.573ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d") failed: Property not found. retry in 73.566063ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d
8032024-12-21T01:14:35.655ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d") failed: Property not found. retry in 117.297119ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d
8042024-12-21T01:14:35.785ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d") failed: Property not found. retry in 335.828521ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d
8052024-12-21T01:14:36.130ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d") failed: Property not found. retry in 1.038547671s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d
8062024-12-21T01:14:37.179ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d") failed: Property not found. retry in 1.384562543s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d
8072024-12-21T01:14:38.135ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 1912036175591875858 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
8082024-12-21T01:14:38.135ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 576692204050020974 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
8092024-12-21T01:14:38.573ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d") failed: Property not found. retry in 1.226327157s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d
8102024-12-21T01:14:39.504ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 11062531147977962647 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
8112024-12-21T01:14:39.504ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 14223862254712097596 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
8122024-12-21T01:14:39.809ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d") failed: Property not found. retry in 562.379173ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d
8132024-12-21T01:14:40.392ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d
8142024-12-21T01:14:40.392ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
8152024-12-21T01:14:40.392ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
8162024-12-21T01:14:40.393ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5802054 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:63465 req_id = e9013025-d8b7-46e3-82b5-402871d0bca3 response_code = 204 uri = /omicron-zones
8172024-12-21T01:14:40.395ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:735
8182024-12-21T01:14:40.426ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8192024-12-21T01:14:40.426ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:56214
8202024-12-21T01:14:40.436ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3915
8212024-12-21T01:14:40.437ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 10145 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:56214 req_id = fd2889ad-e1e6-40f2-aae1-98f3595d45bc response_code = 500 uri = /timesync
8222024-12-21T01:14:40.440ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8232024-12-21T01:14:40.594ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8242024-12-21T01:14:40.594ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60639
8252024-12-21T01:14:43.370ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 5355691491720831998 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
8262024-12-21T01:14:43.929ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 3848167077148183257 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
8272024-12-21T01:14:44.021ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 1950196078331827126 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
8282024-12-21T01:14:44.603ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3915
8292024-12-21T01:14:44.603ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 4008236 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:60639 req_id = ad7eb081-c5d9-4385-b7ef-950ef62ca827 response_code = 500 uri = /timesync
8302024-12-21T01:14:44.605ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8312024-12-21T01:14:44.757ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8322024-12-21T01:14:44.757ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:45699
8332024-12-21T01:14:44.766ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3915
8342024-12-21T01:14:44.766ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 9105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:45699 req_id = 43e2b210-116d-428f-a1c0-4916624f617e response_code = 500 uri = /timesync
8352024-12-21T01:14:44.769ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8362024-12-21T01:14:44.967ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8372024-12-21T01:14:44.967ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:40410
8382024-12-21T01:14:44.975ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_f1bcff64-b896-4076-8a2b-587d701f235d': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3915
8392024-12-21T01:14:44.975ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 8159 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:40410 req_id = a376324a-5eed-4d77-ba72-21d7d9382afb response_code = 500 uri = /timesync
8402024-12-21T01:14:44.978ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8412024-12-21T01:14:45.164ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8422024-12-21T01:14:45.164ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50487
8432024-12-21T01:14:48.137ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 1912036175591875858 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
8442024-12-21T01:14:48.137ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 576692204050020974 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
8452024-12-21T01:14:49.172ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4007935 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:50487 req_id = 3efb2199-0c08-4010-a41e-6e8c73fda40e response_code = 200 uri = /timesync
8462024-12-21T01:14:49.175ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 2139029761, ip_addr: ::, stratum: 10, ref_time: 1734743687.275597, correction: 0.027311562 } file = sled-agent/src/rack_setup/service.rs:743
8472024-12-21T01:14:49.175ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8482024-12-21T01:14:49.488ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8492024-12-21T01:14:49.488ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:40747
8502024-12-21T01:14:49.508ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 11062531147977962647 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
8512024-12-21T01:14:49.508ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 14223862254712097596 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
8522024-12-21T01:14:49.509ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 20147 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:40747 req_id = 895f4966-55bf-4e2d-84d2-0da08f5fc4df response_code = 200 uri = /timesync
8532024-12-21T01:14:49.511ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 2180654075, ip_addr: 129.250.35.251, stratum: 3, ref_time: 1734743689.2272513, correction: 0.027909268 } file = sled-agent/src/rack_setup/service.rs:743
8542024-12-21T01:14:49.511ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1392
8552024-12-21T01:14:49.542ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: d757960c-713d-487d-afb1-254f04de69f7 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 8e844631-8283-4702-ac45-290bbf4afea5 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 76b38c45-7399-47de-bb68-255b313cfd9f (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: f1bcff64-b896-4076-8a2b-587d701f235d (service), filesystem_pool: Some(ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: edefca09-f432-47bc-95dc-35286f50fcc4, kind: Service { id: f1bcff64-b896-4076-8a2b-587d701f235d }, name: Name("ntp-f1bcff64-b896-4076-8a2b-587d701f235d"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 1f40b476-2b7b-40b6-b80c-448a7f2f6719 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } } } }, OmicronZoneConfig { id: 05be4ec5-2c04-4f5f-8702-7c930bbe2842 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 52c0dc4d-cd83-4d35-86a3-0d1a5431963b (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1ac9df71-763a-4041-920b-9a9b6530bc36 (service), filesystem_pool: Some(ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External } } } }, OmicronZoneConfig { id: a2649a40-3966-4efd-a88c-8155847c1261 (service), filesystem_pool: Some(ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
8562024-12-21T01:14:49.542ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64940
8572024-12-21T01:14:49.542ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
8582024-12-21T01:14:49.542ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }
8592024-12-21T01:14:49.554ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
8602024-12-21T01:14:49.554ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8612024-12-21T01:14:49.563ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
8622024-12-21T01:14:49.563ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8632024-12-21T01:14:49.572ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
8642024-12-21T01:14:49.572ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }
8652024-12-21T01:14:49.582ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
8662024-12-21T01:14:49.582ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8672024-12-21T01:14:49.590ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
8682024-12-21T01:14:49.590ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8692024-12-21T01:14:49.600ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
8702024-12-21T01:14:49.600ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8712024-12-21T01:14:49.608ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
8722024-12-21T01:14:49.609ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8732024-12-21T01:14:49.617ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
8742024-12-21T01:14:49.617ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-zones.json file = common/src/ledger.rs:177
8752024-12-21T01:14:49.618ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-zones.json file = common/src/ledger.rs:177
8762024-12-21T01:14:49.618ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
8772024-12-21T01:14:49.649ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
8782024-12-21T01:14:49.649ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
8792024-12-21T01:14:49.649ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
8802024-12-21T01:14:49.649ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
8812024-12-21T01:14:49.649ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
8822024-12-21T01:14:49.649ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842", TimeNotSynchronized), ("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261", TimeNotSynchronized), ("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b", TimeNotSynchronized), ("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719", TimeNotSynchronized), ("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 106735 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:64940 req_id = fc4c6c70-5861-4f13-80da-5f637a110ec5 response_code = 500 uri = /omicron-zones
8832024-12-21T01:14:49.649ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "fc4c6c70-5861-4f13-80da-5f637a110ec5", "content-length": "124", "date": "Sat, 21 Dec 2024 01:14:49 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "fc4c6c70-5861-4f13-80da-5f637a110ec5" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 76.444879ms sled_address = [fd00:1122:3344:101::1]:12345
8842024-12-21T01:14:49.728ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: d757960c-713d-487d-afb1-254f04de69f7 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 8e844631-8283-4702-ac45-290bbf4afea5 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 76b38c45-7399-47de-bb68-255b313cfd9f (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: f1bcff64-b896-4076-8a2b-587d701f235d (service), filesystem_pool: Some(ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: edefca09-f432-47bc-95dc-35286f50fcc4, kind: Service { id: f1bcff64-b896-4076-8a2b-587d701f235d }, name: Name("ntp-f1bcff64-b896-4076-8a2b-587d701f235d"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 1f40b476-2b7b-40b6-b80c-448a7f2f6719 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } } } }, OmicronZoneConfig { id: 05be4ec5-2c04-4f5f-8702-7c930bbe2842 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 52c0dc4d-cd83-4d35-86a3-0d1a5431963b (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1ac9df71-763a-4041-920b-9a9b6530bc36 (service), filesystem_pool: Some(ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External } } } }, OmicronZoneConfig { id: a2649a40-3966-4efd-a88c-8155847c1261 (service), filesystem_pool: Some(ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
8852024-12-21T01:14:49.728ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
8862024-12-21T01:14:49.728ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }
8872024-12-21T01:14:49.739ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
8882024-12-21T01:14:49.739ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8892024-12-21T01:14:49.747ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
8902024-12-21T01:14:49.747ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8912024-12-21T01:14:49.756ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
8922024-12-21T01:14:49.756ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }
8932024-12-21T01:14:49.764ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
8942024-12-21T01:14:49.764ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8952024-12-21T01:14:49.774ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
8962024-12-21T01:14:49.774ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8972024-12-21T01:14:49.784ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
8982024-12-21T01:14:49.784ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8992024-12-21T01:14:49.793ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9002024-12-21T01:14:49.793ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9012024-12-21T01:14:49.802ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9022024-12-21T01:14:49.802ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-zones.json file = common/src/ledger.rs:177
9032024-12-21T01:14:49.802ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-zones.json file = common/src/ledger.rs:177
9042024-12-21T01:14:49.802ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9052024-12-21T01:14:49.833ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
9062024-12-21T01:14:49.833ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
9072024-12-21T01:14:49.833ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
9082024-12-21T01:14:49.833ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
9092024-12-21T01:14:49.833ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
9102024-12-21T01:14:49.833ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261", TimeNotSynchronized), ("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b", TimeNotSynchronized), ("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719", TimeNotSynchronized), ("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36", TimeNotSynchronized), ("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 104878 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:64940 req_id = aa7bf756-d92a-426d-b53e-46d4c42f63d1 response_code = 500 uri = /omicron-zones
9112024-12-21T01:14:49.833ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "aa7bf756-d92a-426d-b53e-46d4c42f63d1", "content-length": "124", "date": "Sat, 21 Dec 2024 01:14:49 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "aa7bf756-d92a-426d-b53e-46d4c42f63d1" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 123.816391ms sled_address = [fd00:1122:3344:101::1]:12345
9122024-12-21T01:14:49.958ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: d757960c-713d-487d-afb1-254f04de69f7 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 8e844631-8283-4702-ac45-290bbf4afea5 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 76b38c45-7399-47de-bb68-255b313cfd9f (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: f1bcff64-b896-4076-8a2b-587d701f235d (service), filesystem_pool: Some(ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: edefca09-f432-47bc-95dc-35286f50fcc4, kind: Service { id: f1bcff64-b896-4076-8a2b-587d701f235d }, name: Name("ntp-f1bcff64-b896-4076-8a2b-587d701f235d"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 1f40b476-2b7b-40b6-b80c-448a7f2f6719 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } } } }, OmicronZoneConfig { id: 05be4ec5-2c04-4f5f-8702-7c930bbe2842 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 52c0dc4d-cd83-4d35-86a3-0d1a5431963b (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1ac9df71-763a-4041-920b-9a9b6530bc36 (service), filesystem_pool: Some(ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External } } } }, OmicronZoneConfig { id: a2649a40-3966-4efd-a88c-8155847c1261 (service), filesystem_pool: Some(ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
9132024-12-21T01:14:49.958ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9142024-12-21T01:14:49.958ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }
9152024-12-21T01:14:49.968ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9162024-12-21T01:14:49.968ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9172024-12-21T01:14:49.975ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9182024-12-21T01:14:49.976ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9192024-12-21T01:14:49.984ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9202024-12-21T01:14:49.984ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }
9212024-12-21T01:14:49.992ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9222024-12-21T01:14:49.992ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9232024-12-21T01:14:50.002ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9242024-12-21T01:14:50.002ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9252024-12-21T01:14:50.011ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9262024-12-21T01:14:50.011ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9272024-12-21T01:14:50.021ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9282024-12-21T01:14:50.021ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9292024-12-21T01:14:50.030ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9302024-12-21T01:14:50.030ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-zones.json file = common/src/ledger.rs:177
9312024-12-21T01:14:50.030ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-zones.json file = common/src/ledger.rs:177
9322024-12-21T01:14:50.030ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9332024-12-21T01:14:50.061ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
9342024-12-21T01:14:50.061ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
9352024-12-21T01:14:50.061ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
9362024-12-21T01:14:50.061ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
9372024-12-21T01:14:50.061ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
9382024-12-21T01:14:50.061ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842", TimeNotSynchronized), ("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b", TimeNotSynchronized), ("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36", TimeNotSynchronized), ("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261", TimeNotSynchronized), ("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 102538 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:64940 req_id = 68c0f33c-557e-435f-a020-97a2231e2e49 response_code = 500 uri = /omicron-zones
9392024-12-21T01:14:50.061ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "68c0f33c-557e-435f-a020-97a2231e2e49", "content-length": "124", "date": "Sat, 21 Dec 2024 01:14:49 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "68c0f33c-557e-435f-a020-97a2231e2e49" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 166.472263ms sled_address = [fd00:1122:3344:101::1]:12345
9402024-12-21T01:14:50.229ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: d757960c-713d-487d-afb1-254f04de69f7 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 8e844631-8283-4702-ac45-290bbf4afea5 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 76b38c45-7399-47de-bb68-255b313cfd9f (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: f1bcff64-b896-4076-8a2b-587d701f235d (service), filesystem_pool: Some(ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: edefca09-f432-47bc-95dc-35286f50fcc4, kind: Service { id: f1bcff64-b896-4076-8a2b-587d701f235d }, name: Name("ntp-f1bcff64-b896-4076-8a2b-587d701f235d"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 1f40b476-2b7b-40b6-b80c-448a7f2f6719 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } } } }, OmicronZoneConfig { id: 05be4ec5-2c04-4f5f-8702-7c930bbe2842 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 52c0dc4d-cd83-4d35-86a3-0d1a5431963b (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1ac9df71-763a-4041-920b-9a9b6530bc36 (service), filesystem_pool: Some(ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External } } } }, OmicronZoneConfig { id: a2649a40-3966-4efd-a88c-8155847c1261 (service), filesystem_pool: Some(ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
9412024-12-21T01:14:50.229ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9422024-12-21T01:14:50.229ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }
9432024-12-21T01:14:50.238ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9442024-12-21T01:14:50.238ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9452024-12-21T01:14:50.247ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9462024-12-21T01:14:50.247ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9472024-12-21T01:14:50.255ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9482024-12-21T01:14:50.255ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }
9492024-12-21T01:14:50.265ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9502024-12-21T01:14:50.265ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9512024-12-21T01:14:50.274ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9522024-12-21T01:14:50.274ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9532024-12-21T01:14:50.282ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9542024-12-21T01:14:50.282ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9552024-12-21T01:14:50.292ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9562024-12-21T01:14:50.292ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9572024-12-21T01:14:50.301ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9582024-12-21T01:14:50.301ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-zones.json file = common/src/ledger.rs:177
9592024-12-21T01:14:50.301ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-zones.json file = common/src/ledger.rs:177
9602024-12-21T01:14:50.301ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9612024-12-21T01:14:50.333ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
9622024-12-21T01:14:50.333ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
9632024-12-21T01:14:50.333ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
9642024-12-21T01:14:50.333ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
9652024-12-21T01:14:50.333ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
9662024-12-21T01:14:50.333ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719", TimeNotSynchronized), ("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36", TimeNotSynchronized), ("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842", TimeNotSynchronized), ("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261", TimeNotSynchronized), ("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 103539 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:64940 req_id = 86f9a80d-114c-4758-bf5f-f63cf6828250 response_code = 500 uri = /omicron-zones
9672024-12-21T01:14:50.333ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "86f9a80d-114c-4758-bf5f-f63cf6828250", "content-length": "124", "date": "Sat, 21 Dec 2024 01:14:50 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "86f9a80d-114c-4758-bf5f-f63cf6828250" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 206.541626ms sled_address = [fd00:1122:3344:101::1]:12345
9682024-12-21T01:14:50.542ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: d757960c-713d-487d-afb1-254f04de69f7 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 8e844631-8283-4702-ac45-290bbf4afea5 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 76b38c45-7399-47de-bb68-255b313cfd9f (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: f1bcff64-b896-4076-8a2b-587d701f235d (service), filesystem_pool: Some(ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: edefca09-f432-47bc-95dc-35286f50fcc4, kind: Service { id: f1bcff64-b896-4076-8a2b-587d701f235d }, name: Name("ntp-f1bcff64-b896-4076-8a2b-587d701f235d"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 1f40b476-2b7b-40b6-b80c-448a7f2f6719 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } } } }, OmicronZoneConfig { id: 05be4ec5-2c04-4f5f-8702-7c930bbe2842 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 52c0dc4d-cd83-4d35-86a3-0d1a5431963b (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1ac9df71-763a-4041-920b-9a9b6530bc36 (service), filesystem_pool: Some(ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External } } } }, OmicronZoneConfig { id: a2649a40-3966-4efd-a88c-8155847c1261 (service), filesystem_pool: Some(ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
9692024-12-21T01:14:50.542ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9702024-12-21T01:14:50.542ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }
9712024-12-21T01:14:50.552ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9722024-12-21T01:14:50.552ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9732024-12-21T01:14:50.561ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9742024-12-21T01:14:50.561ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9752024-12-21T01:14:50.570ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9762024-12-21T01:14:50.570ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }
9772024-12-21T01:14:50.580ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9782024-12-21T01:14:50.580ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9792024-12-21T01:14:50.589ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9802024-12-21T01:14:50.589ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9812024-12-21T01:14:50.597ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9822024-12-21T01:14:50.597ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9832024-12-21T01:14:50.605ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
9842024-12-21T01:14:50.605ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9852024-12-21T01:14:50.614ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9862024-12-21T01:14:50.614ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-zones.json file = common/src/ledger.rs:177
9872024-12-21T01:14:50.614ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-zones.json file = common/src/ledger.rs:177
9882024-12-21T01:14:50.614ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
9892024-12-21T01:14:50.646ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
9902024-12-21T01:14:50.646ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
9912024-12-21T01:14:50.646ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
9922024-12-21T01:14:50.646ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
9932024-12-21T01:14:50.646ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
9942024-12-21T01:14:50.646ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719", TimeNotSynchronized), ("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b", TimeNotSynchronized), ("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36", TimeNotSynchronized), ("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842", TimeNotSynchronized), ("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 103620 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:64940 req_id = 9c8dafee-7aa8-48ea-8caa-fb1c8801634a response_code = 500 uri = /omicron-zones
9952024-12-21T01:14:50.646ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "9c8dafee-7aa8-48ea-8caa-fb1c8801634a", "content-length": "124", "date": "Sat, 21 Dec 2024 01:14:50 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "9c8dafee-7aa8-48ea-8caa-fb1c8801634a" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 155.731255ms sled_address = [fd00:1122:3344:101::1]:12345
9962024-12-21T01:14:50.804ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: d757960c-713d-487d-afb1-254f04de69f7 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 8e844631-8283-4702-ac45-290bbf4afea5 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 76b38c45-7399-47de-bb68-255b313cfd9f (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: f1bcff64-b896-4076-8a2b-587d701f235d (service), filesystem_pool: Some(ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: edefca09-f432-47bc-95dc-35286f50fcc4, kind: Service { id: f1bcff64-b896-4076-8a2b-587d701f235d }, name: Name("ntp-f1bcff64-b896-4076-8a2b-587d701f235d"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 1f40b476-2b7b-40b6-b80c-448a7f2f6719 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } } } }, OmicronZoneConfig { id: 05be4ec5-2c04-4f5f-8702-7c930bbe2842 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 52c0dc4d-cd83-4d35-86a3-0d1a5431963b (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1ac9df71-763a-4041-920b-9a9b6530bc36 (service), filesystem_pool: Some(ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External } } } }, OmicronZoneConfig { id: a2649a40-3966-4efd-a88c-8155847c1261 (service), filesystem_pool: Some(ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
9972024-12-21T01:14:50.804ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
9982024-12-21T01:14:50.804ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }
9992024-12-21T01:14:50.814ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10002024-12-21T01:14:50.814ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10012024-12-21T01:14:50.823ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10022024-12-21T01:14:50.823ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10032024-12-21T01:14:50.832ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10042024-12-21T01:14:50.832ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }
10052024-12-21T01:14:50.840ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10062024-12-21T01:14:50.840ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10072024-12-21T01:14:50.849ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10082024-12-21T01:14:50.849ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10092024-12-21T01:14:50.858ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10102024-12-21T01:14:50.859ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10112024-12-21T01:14:50.867ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10122024-12-21T01:14:50.867ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10132024-12-21T01:14:50.876ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10142024-12-21T01:14:50.876ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-zones.json file = common/src/ledger.rs:177
10152024-12-21T01:14:50.877ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-zones.json file = common/src/ledger.rs:177
10162024-12-21T01:14:50.877ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10172024-12-21T01:14:50.910ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
10182024-12-21T01:14:50.910ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
10192024-12-21T01:14:50.910ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
10202024-12-21T01:14:50.910ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
10212024-12-21T01:14:50.910ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
10222024-12-21T01:14:50.910ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719", TimeNotSynchronized), ("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842", TimeNotSynchronized), ("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261", TimeNotSynchronized), ("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b", TimeNotSynchronized), ("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 106027 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:64940 req_id = f8dd3337-4670-456b-807b-2ce57cdf853e response_code = 500 uri = /omicron-zones
10232024-12-21T01:14:50.911ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "f8dd3337-4670-456b-807b-2ce57cdf853e", "content-length": "124", "date": "Sat, 21 Dec 2024 01:14:50 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "f8dd3337-4670-456b-807b-2ce57cdf853e" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 198.527376ms sled_address = [fd00:1122:3344:101::1]:12345
10242024-12-21T01:14:51.112ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: d757960c-713d-487d-afb1-254f04de69f7 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 8e844631-8283-4702-ac45-290bbf4afea5 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 76b38c45-7399-47de-bb68-255b313cfd9f (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: f1bcff64-b896-4076-8a2b-587d701f235d (service), filesystem_pool: Some(ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: edefca09-f432-47bc-95dc-35286f50fcc4, kind: Service { id: f1bcff64-b896-4076-8a2b-587d701f235d }, name: Name("ntp-f1bcff64-b896-4076-8a2b-587d701f235d"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 1f40b476-2b7b-40b6-b80c-448a7f2f6719 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } } } }, OmicronZoneConfig { id: 05be4ec5-2c04-4f5f-8702-7c930bbe2842 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 52c0dc4d-cd83-4d35-86a3-0d1a5431963b (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1ac9df71-763a-4041-920b-9a9b6530bc36 (service), filesystem_pool: Some(ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External } } } }, OmicronZoneConfig { id: a2649a40-3966-4efd-a88c-8155847c1261 (service), filesystem_pool: Some(ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
10252024-12-21T01:14:51.112ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10262024-12-21T01:14:51.112ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }
10272024-12-21T01:14:51.122ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10282024-12-21T01:14:51.122ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10292024-12-21T01:14:51.131ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10302024-12-21T01:14:51.131ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10312024-12-21T01:14:51.140ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10322024-12-21T01:14:51.140ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }
10332024-12-21T01:14:51.150ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10342024-12-21T01:14:51.150ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10352024-12-21T01:14:51.158ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10362024-12-21T01:14:51.158ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10372024-12-21T01:14:51.166ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10382024-12-21T01:14:51.166ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10392024-12-21T01:14:51.175ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10402024-12-21T01:14:51.175ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10412024-12-21T01:14:51.184ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10422024-12-21T01:14:51.184ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-zones.json file = common/src/ledger.rs:177
10432024-12-21T01:14:51.184ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-zones.json file = common/src/ledger.rs:177
10442024-12-21T01:14:51.184ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10452024-12-21T01:14:51.216ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
10462024-12-21T01:14:51.216ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
10472024-12-21T01:14:51.216ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
10482024-12-21T01:14:51.216ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
10492024-12-21T01:14:51.216ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3329 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
10502024-12-21T01:14:51.216ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719", TimeNotSynchronized), ("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36", TimeNotSynchronized), ("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b", TimeNotSynchronized), ("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842", TimeNotSynchronized), ("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 103996 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:64940 req_id = f3ea8cca-b5d1-4a47-be4d-5dec0d5b9efd response_code = 500 uri = /omicron-zones
10512024-12-21T01:14:51.216ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "f3ea8cca-b5d1-4a47-be4d-5dec0d5b9efd", "content-length": "124", "date": "Sat, 21 Dec 2024 01:14:50 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "f3ea8cca-b5d1-4a47-be4d-5dec0d5b9efd" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 360.90226ms sled_address = [fd00:1122:3344:101::1]:12345
10522024-12-21T01:14:51.580ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: d757960c-713d-487d-afb1-254f04de69f7 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 8e844631-8283-4702-ac45-290bbf4afea5 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 76b38c45-7399-47de-bb68-255b313cfd9f (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: f1bcff64-b896-4076-8a2b-587d701f235d (service), filesystem_pool: Some(ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: edefca09-f432-47bc-95dc-35286f50fcc4, kind: Service { id: f1bcff64-b896-4076-8a2b-587d701f235d }, name: Name("ntp-f1bcff64-b896-4076-8a2b-587d701f235d"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 1f40b476-2b7b-40b6-b80c-448a7f2f6719 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } } } }, OmicronZoneConfig { id: 05be4ec5-2c04-4f5f-8702-7c930bbe2842 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 52c0dc4d-cd83-4d35-86a3-0d1a5431963b (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1ac9df71-763a-4041-920b-9a9b6530bc36 (service), filesystem_pool: Some(ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External } } } }, OmicronZoneConfig { id: a2649a40-3966-4efd-a88c-8155847c1261 (service), filesystem_pool: Some(ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
10532024-12-21T01:14:51.580ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10542024-12-21T01:14:51.580ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }
10552024-12-21T01:14:51.590ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10562024-12-21T01:14:51.590ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10572024-12-21T01:14:51.598ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
10582024-12-21T01:14:51.598ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10592024-12-21T01:14:51.608ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10602024-12-21T01:14:51.608ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }
10612024-12-21T01:14:51.616ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10622024-12-21T01:14:51.616ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10632024-12-21T01:14:51.625ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10642024-12-21T01:14:51.625ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10652024-12-21T01:14:51.635ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10662024-12-21T01:14:51.635ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10672024-12-21T01:14:51.644ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
10682024-12-21T01:14:51.644ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10692024-12-21T01:14:51.652ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10702024-12-21T01:14:51.653ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-zones.json file = common/src/ledger.rs:177
10712024-12-21T01:14:51.653ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-zones.json file = common/src/ledger.rs:177
10722024-12-21T01:14:51.653ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10732024-12-21T01:14:51.735ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10742024-12-21T01:14:51.736ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10752024-12-21T01:14:51.736ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10762024-12-21T01:14:51.736ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10772024-12-21T01:14:51.736ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
10782024-12-21T01:14:51.827ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b file = illumos-utils/src/zone.rs:325
10792024-12-21T01:14:51.831ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842 file = illumos-utils/src/zone.rs:325
10802024-12-21T01:14:51.835ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36 file = illumos-utils/src/zone.rs:325
10812024-12-21T01:14:51.839ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261 file = illumos-utils/src/zone.rs:325
10822024-12-21T01:14:51.844ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719 file = illumos-utils/src/zone.rs:325
10832024-12-21T01:14:51.848ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b file = illumos-utils/src/zone.rs:363
10842024-12-21T01:14:51.853ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842 file = illumos-utils/src/zone.rs:363
10852024-12-21T01:14:51.857ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36 file = illumos-utils/src/zone.rs:363
10862024-12-21T01:14:51.862ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261 file = illumos-utils/src/zone.rs:363
10872024-12-21T01:14:51.867ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719 file = illumos-utils/src/zone.rs:363
10882024-12-21T01:14:52.650ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 364.622232ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
10892024-12-21T01:14:55.471ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService5'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='52c0dc4d-cd83-4d35-86a3-0d1a5431963b'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::5]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10902024-12-21T01:14:55.473ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
10912024-12-21T01:14:55.764ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService7'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='1ac9df71-763a-4041-920b-9a9b6530bc36'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::6]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10922024-12-21T01:14:55.764ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
10932024-12-21T01:14:55.765ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService8'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='a2649a40-3966-4efd-a88c-8155847c1261'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::7]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10942024-12-21T01:14:55.765ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService9'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::3'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='1f40b476-2b7b-40b6-b80c-448a7f2f6719'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::3]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10952024-12-21T01:14:55.765ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
10962024-12-21T01:14:55.767ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
10972024-12-21T01:14:56.008ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService6'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::4'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='05be4ec5-2c04-4f5f-8702-7c930bbe2842'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::4]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
10982024-12-21T01:14:56.008ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
10992024-12-21T01:14:56.065ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36") failed: Property not found. retry in 53.311934ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
11002024-12-21T01:14:56.074ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 53.525513ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11012024-12-21T01:14:56.086ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261") failed: Property not found. retry in 44.046847ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
11022024-12-21T01:14:56.096ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719") failed: Property not found. retry in 43.195699ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
11032024-12-21T01:14:56.134ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36") failed: Property not found. retry in 50.50833ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
11042024-12-21T01:14:56.141ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 76.604654ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11052024-12-21T01:14:56.153ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261") failed: Property not found. retry in 115.790147ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
11062024-12-21T01:14:56.169ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719") failed: Property not found. retry in 130.51286ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
11072024-12-21T01:14:56.202ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36") failed: Property not found. retry in 289.48618ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
11082024-12-21T01:14:56.231ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 244.881892ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11092024-12-21T01:14:56.241ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 37.249635ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11102024-12-21T01:14:56.287ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261") failed: Property not found. retry in 293.22358ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
11112024-12-21T01:14:56.315ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 92.177228ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11122024-12-21T01:14:56.335ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719") failed: Property not found. retry in 183.084746ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
11132024-12-21T01:14:56.419ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 263.420729ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11142024-12-21T01:14:56.492ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 417.241569ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11152024-12-21T01:14:56.507ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36") failed: Property not found. retry in 360.014145ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
11162024-12-21T01:14:56.530ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719") failed: Property not found. retry in 332.43963ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
11172024-12-21T01:14:56.592ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261") failed: Property not found. retry in 423.601398ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
11182024-12-21T01:14:56.698ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 572.119956ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11192024-12-21T01:14:56.876ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719") failed: Property not found. retry in 743.935565ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
11202024-12-21T01:14:56.886ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36") failed: Property not found. retry in 1.156333008s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
11212024-12-21T01:14:56.923ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 759.210597ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11222024-12-21T01:14:57.030ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261") failed: Property not found. retry in 839.923544ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
11232024-12-21T01:14:57.284ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 1.082965148s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11242024-12-21T01:14:57.634ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719") failed: Property not found. retry in 1.190336687s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
11252024-12-21T01:14:57.695ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 1.340866767s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11262024-12-21T01:14:57.881ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261") failed: Property not found. retry in 837.604339ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
11272024-12-21T01:14:58.054ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36") failed: Property not found. retry in 927.831571ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
11282024-12-21T01:14:58.380ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 1.318488805s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11292024-12-21T01:14:58.609ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:60313
11302024-12-21T01:14:58.610ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:60313 req_id = eb203a4f-66a8-4baf-a3e8-8bcf1ca4c94b response_code = 200 uri = /baseboard
11312024-12-21T01:14:58.731ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261") failed: Property not found. retry in 689.393857ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
11322024-12-21T01:14:58.836ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719") failed: Property not found. retry in 1.401312625s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
11332024-12-21T01:14:58.992ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36") failed: Property not found. retry in 812.089772ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
11342024-12-21T01:14:59.092ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 1.068509665s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11352024-12-21T01:14:59.433ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261") failed: Property not found. retry in 1.039333313s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
11362024-12-21T01:14:59.714ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 846.654907ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11372024-12-21T01:14:59.818ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36") failed: Property not found. retry in 1.141897345s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
11382024-12-21T01:15:00.177ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 636.027435ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11392024-12-21T01:15:00.252ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719") failed: Property not found. retry in 829.973024ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
11402024-12-21T01:15:00.484ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261") failed: Property not found. retry in 911.968042ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
11412024-12-21T01:15:00.571ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 1.093462082s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11422024-12-21T01:15:00.824ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 667.321844ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11432024-12-21T01:15:00.970ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36") failed: Property not found. retry in 1.0328403s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
11442024-12-21T01:15:01.091ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719") failed: Property not found. retry in 561.563586ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
11452024-12-21T01:15:01.405ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261") failed: Property not found. retry in 761.201181ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
11462024-12-21T01:15:01.498ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 1.489771963s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11472024-12-21T01:15:01.661ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719") failed: Property not found. retry in 1.244922051s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
11482024-12-21T01:15:01.673ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 931.099639ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11492024-12-21T01:15:02.013ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36") failed: Property not found. retry in 1.097167865s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
11502024-12-21T01:15:02.176ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261") failed: Property not found. retry in 1.101637965s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
11512024-12-21T01:15:02.614ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 779.658141ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11522024-12-21T01:15:02.915ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719") failed: Property not found. retry in 1.375773136s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
11532024-12-21T01:15:02.998ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 589.563959ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11542024-12-21T01:15:03.120ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36") failed: Property not found. retry in 955.814276ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
11552024-12-21T01:15:03.404ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 1.196699442s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11562024-12-21T01:15:03.599ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 901.47542ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11572024-12-21T01:15:04.510ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 867.901036ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11582024-12-21T01:15:04.610ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 1.106107297s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11592024-12-21T01:15:05.389ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 608.76342ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11602024-12-21T01:15:05.726ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 833.947165ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11612024-12-21T01:15:06.012ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 752.175058ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11622024-12-21T01:15:06.767ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 910.269549ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11632024-12-21T01:15:06.790ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 1.28700842s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11642024-12-21T01:15:07.693ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 826.357667ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11652024-12-21T01:15:08.068ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 391.813617ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
11662024-12-21T01:15:08.090ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 1.39913296s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11672024-12-21T01:15:08.530ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 1.071421961s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11682024-12-21T01:15:09.498ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 1.442916865s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11692024-12-21T01:15:09.610ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 1.013469741s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11702024-12-21T01:15:10.634ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 1.476050547s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11712024-12-21T01:15:10.950ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 572.314763ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11722024-12-21T01:15:11.531ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 1.041987072s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11732024-12-21T01:15:12.118ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 811.133338ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11742024-12-21T01:15:12.583ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b") failed: Property not found. retry in 1.273298047s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11752024-12-21T01:15:12.939ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842") failed: Property not found. retry in 662.260492ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11762024-12-21T01:15:13.875ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_cockroachdb_52c0dc4d-cd83-4d35-86a3-0d1a5431963b
11772024-12-21T01:15:13.875ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842
11782024-12-21T01:15:13.875ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_cockroachdb_1ac9df71-763a-4041-920b-9a9b6530bc36
11792024-12-21T01:15:13.875ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_cockroachdb_a2649a40-3966-4efd-a88c-8155847c1261
11802024-12-21T01:15:13.875ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_cockroachdb_1f40b476-2b7b-40b6-b80c-448a7f2f6719
11812024-12-21T01:15:13.875ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
11822024-12-21T01:15:13.876ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
11832024-12-21T01:15:13.877ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 22274431 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:64940 req_id = 302570bd-e21c-44bb-ba06-8281471f6b5a response_code = 204 uri = /omicron-zones
11842024-12-21T01:15:13.910ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50411
11852024-12-21T01:15:13.938ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 27412 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:50411 req_id = d5db7d1a-b9f4-497c-a259-6b2724173ad6 response_code = 500 uri = /cockroachdb
11862024-12-21T01:15:13.938ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "d5db7d1a-b9f4-497c-a259-6b2724173ad6", "content-length": "124", "date": "Sat, 21 Dec 2024 01:15:13 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "d5db7d1a-b9f4-497c-a259-6b2724173ad6" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 146.797591ms
11872024-12-21T01:15:14.112ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 26519 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:50411 req_id = 6b713e2c-00b8-44ff-a997-34e95c968889 response_code = 500 uri = /cockroachdb
11882024-12-21T01:15:14.112ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "6b713e2c-00b8-44ff-a997-34e95c968889", "content-length": "124", "date": "Sat, 21 Dec 2024 01:15:13 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "6b713e2c-00b8-44ff-a997-34e95c968889" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 69.542913ms
11892024-12-21T01:15:14.206ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService6/omicron6 in oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_05be4ec5-2c04-4f5f-8702-7c930bbe2842 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService6/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 23759 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:50411 req_id = 315e1001-dd2f-4325-b1ed-785978ba23c3 response_code = 500 uri = /cockroachdb
11902024-12-21T01:15:14.206ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "315e1001-dd2f-4325-b1ed-785978ba23c3", "content-length": "124", "date": "Sat, 21 Dec 2024 01:15:13 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "315e1001-dd2f-4325-b1ed-785978ba23c3" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 100.801234ms
11912024-12-21T01:15:14.337ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::4]:32221 file = sled-agent/src/services.rs:3758
11922024-12-21T01:15:20.212ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3776
11932024-12-21T01:15:36.201ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:3799
11942024-12-21T01:15:36.202ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 21892842 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:50411 req_id = 4674213b-d6d6-455d-85d4-26d706c3e952 response_code = 204 uri = /cockroachdb
11952024-12-21T01:15:36.202ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:52671
11962024-12-21T01:15:36.203ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 154 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:52671 req_id = 12137d8a-1f97-41e0-9dcf-fea434bb2010 response_code = 200 uri = /baseboard
11972024-12-21T01:15:36.205ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 1.36346666s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
11982024-12-21T01:15:36.236ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: d757960c-713d-487d-afb1-254f04de69f7 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 8e844631-8283-4702-ac45-290bbf4afea5 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: 76b38c45-7399-47de-bb68-255b313cfd9f (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: f1bcff64-b896-4076-8a2b-587d701f235d (service), filesystem_pool: Some(ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: edefca09-f432-47bc-95dc-35286f50fcc4, kind: Service { id: f1bcff64-b896-4076-8a2b-587d701f235d }, name: Name("ntp-f1bcff64-b896-4076-8a2b-587d701f235d"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 134, 39])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: 1f40b476-2b7b-40b6-b80c-448a7f2f6719 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } } } }, OmicronZoneConfig { id: 05be4ec5-2c04-4f5f-8702-7c930bbe2842 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 52c0dc4d-cd83-4d35-86a3-0d1a5431963b (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1ac9df71-763a-4041-920b-9a9b6530bc36 (service), filesystem_pool: Some(ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External } } } }, OmicronZoneConfig { id: a2649a40-3966-4efd-a88c-8155847c1261 (service), filesystem_pool: Some(ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External } } } }, OmicronZoneConfig { id: 24428c69-128b-4995-bc7a-addc66912490 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.2.164:53, nic: NetworkInterface { id: cd175b09-5785-415d-8d32-a617204393a9, kind: Service { id: 24428c69-128b-4995-bc7a-addc66912490 }, name: Name("external-dns-24428c69-128b-4995-bc7a-addc66912490"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 190])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: c4364b0f-0b76-413b-9126-391216a47153 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.2.165:53, nic: NetworkInterface { id: 44a38c79-185d-44b1-a294-e3a088dd4f4f, kind: Service { id: c4364b0f-0b76-413b-9126-391216a47153 }, name: Name("external-dns-c4364b0f-0b76-413b-9126-391216a47153"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: ee35abf0-6140-47bd-8a0f-da60e1b0cdad (service), filesystem_pool: Some(ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.2.166, nic: NetworkInterface { id: 5b5216f1-9871-4dfb-aced-f4b109e9a1c2, kind: Service { id: ee35abf0-6140-47bd-8a0f-da60e1b0cdad }, name: Name("nexus-ee35abf0-6140-47bd-8a0f-da60e1b0cdad"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1 (service), filesystem_pool: Some(ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.2.167, nic: NetworkInterface { id: 654f7fda-6e14-4220-bc54-c5c6d8426d7e, kind: Service { id: bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1 }, name: Name("nexus-bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 234, 194])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: fe9c2ca3-0266-4081-952b-4cb3f2bf1371 (service), filesystem_pool: Some(ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.2.168, nic: NetworkInterface { id: 05db974b-0405-4131-ae02-a37152ea980b, kind: Service { id: fe9c2ca3-0266-4081-952b-4cb3f2bf1371 }, name: Name("nexus-fe9c2ca3-0266-4081-952b-4cb3f2bf1371"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 44])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 639157e9-2713-43a8-90e8-75860cd77732 (service), filesystem_pool: Some(ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }), zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: 643661ea-2572-4236-b774-49e58b17e906 (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } } } }, OmicronZoneConfig { id: 6e0a9baf-a85f-4912-82e5-6107854998fc (service), filesystem_pool: Some(ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: 1714c239-c2ea-4fdc-a2ff-2fdc548654f0 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a (service), filesystem_pool: Some(ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: dad237e0-cce5-47d0-a98d-2d417f8b60cf (service), filesystem_pool: Some(ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External } } } }, OmicronZoneConfig { id: 698a7a9e-913f-4d1d-8acf-e625ceef34b1 (service), filesystem_pool: Some(ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External } } } }, OmicronZoneConfig { id: 1519a09c-6c94-4612-b980-ea6d97dcd5c4 (service), filesystem_pool: Some(ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External } } } }, OmicronZoneConfig { id: b387a3bb-a500-4bf1-a8ce-4c3de63bfe21 (service), filesystem_pool: Some(ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External } } } }, OmicronZoneConfig { id: 24749034-6e5c-4b03-9d90-11debee9bb39 (service), filesystem_pool: Some(ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External } } } }, OmicronZoneConfig { id: 6c21be66-efc8-42a3-af47-0191d8ec508d (service), filesystem_pool: Some(ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External } } } }, OmicronZoneConfig { id: b3847803-729f-4a75-bd01-c44489c8d3e4 (service), filesystem_pool: Some(ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External } } } }, OmicronZoneConfig { id: 1f9393ec-0459-42a3-ab95-22a9c9c1839a (service), filesystem_pool: Some(ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External } } } }, OmicronZoneConfig { id: 94b13f11-1aeb-47d6-9e22-e77f14ed68f4 (service), filesystem_pool: Some(ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
11992024-12-21T01:15:36.237ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:65408
12002024-12-21T01:15:36.237ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
12012024-12-21T01:15:36.237ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: InternalDns }, responder: ... }
12022024-12-21T01:15:36.250ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
12032024-12-21T01:15:36.250ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: InternalDns }, responder: ... }
12042024-12-21T01:15:36.261ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
12052024-12-21T01:15:36.261ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: InternalDns }, responder: ... }
12062024-12-21T01:15:36.272ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
12072024-12-21T01:15:36.272ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Cockroach }, responder: ... }
12082024-12-21T01:15:36.281ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
12092024-12-21T01:15:36.281ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Cockroach }, responder: ... }
12102024-12-21T01:15:36.292ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
12112024-12-21T01:15:36.292ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Cockroach }, responder: ... }
12122024-12-21T01:15:36.301ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
12132024-12-21T01:15:36.301ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Cockroach }, responder: ... }
12142024-12-21T01:15:36.310ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:607
12152024-12-21T01:15:36.310ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Cockroach }, responder: ... }
12162024-12-21T01:15:36.318ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
12172024-12-21T01:15:36.318ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: ExternalDns }, responder: ... }
12182024-12-21T01:15:36.329ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:607
12192024-12-21T01:15:36.329ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
12202024-12-21T01:15:36.338ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Clickhouse }, responder: ... }) file = sled-storage/src/manager.rs:607
12212024-12-21T01:15:36.338ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Clickhouse }, responder: ... }
12222024-12-21T01:15:36.352ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12232024-12-21T01:15:36.352ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, kind: Crucible }, responder: ... }
12242024-12-21T01:15:36.367ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12252024-12-21T01:15:36.367ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, kind: Crucible }, responder: ... }
12262024-12-21T01:15:36.376ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12272024-12-21T01:15:36.376ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, kind: Crucible }, responder: ... }
12282024-12-21T01:15:36.387ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12292024-12-21T01:15:36.387ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, kind: Crucible }, responder: ... }
12302024-12-21T01:15:36.398ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12312024-12-21T01:15:36.398ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, kind: Crucible }, responder: ... }
12322024-12-21T01:15:36.410ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12332024-12-21T01:15:36.410ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }, kind: Crucible }, responder: ... }
12342024-12-21T01:15:36.419ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12352024-12-21T01:15:36.419ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }, kind: Crucible }, responder: ... }
12362024-12-21T01:15:36.430ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12372024-12-21T01:15:36.430ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, kind: Crucible }, responder: ... }
12382024-12-21T01:15:36.441ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:607
12392024-12-21T01:15:36.441ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1478 request = NewFilesystemRequest { dataset_id: None, dataset_name: DatasetName { pool_name: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, kind: Crucible }, responder: ... }
12402024-12-21T01:15:36.454ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12412024-12-21T01:15:36.454ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-zones.json file = common/src/ledger.rs:177
12422024-12-21T01:15:36.454ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-zones.json file = common/src/ledger.rs:177
12432024-12-21T01:15:36.454ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12442024-12-21T01:15:36.573ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12452024-12-21T01:15:36.575ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12462024-12-21T01:15:36.588ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12472024-12-21T01:15:36.642ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12482024-12-21T01:15:36.642ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12492024-12-21T01:15:36.642ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12502024-12-21T01:15:36.642ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12512024-12-21T01:15:36.642ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12522024-12-21T01:15:36.642ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12532024-12-21T01:15:36.642ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12542024-12-21T01:15:36.793ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12552024-12-21T01:15:36.793ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12562024-12-21T01:15:36.818ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12572024-12-21T01:15:36.818ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12582024-12-21T01:15:36.818ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12592024-12-21T01:15:36.869ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12602024-12-21T01:15:36.869ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12612024-12-21T01:15:36.869ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12622024-12-21T01:15:36.869ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
12632024-12-21T01:15:36.869ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732 file = illumos-utils/src/zone.rs:325
12642024-12-21T01:15:36.875ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc file = illumos-utils/src/zone.rs:325
12652024-12-21T01:15:36.880ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0 file = illumos-utils/src/zone.rs:325
12662024-12-21T01:15:36.885ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1 file = illumos-utils/src/zone.rs:325
12672024-12-21T01:15:36.890ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4 file = illumos-utils/src/zone.rs:325
12682024-12-21T01:15:36.894ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a file = illumos-utils/src/zone.rs:325
12692024-12-21T01:15:36.899ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf file = illumos-utils/src/zone.rs:325
12702024-12-21T01:15:37.046ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732 file = illumos-utils/src/zone.rs:363
12712024-12-21T01:15:37.051ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc file = illumos-utils/src/zone.rs:363
12722024-12-21T01:15:37.055ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0 file = illumos-utils/src/zone.rs:363
12732024-12-21T01:15:37.059ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1 file = illumos-utils/src/zone.rs:363
12742024-12-21T01:15:37.064ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4 file = illumos-utils/src/zone.rs:363
12752024-12-21T01:15:37.069ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a file = illumos-utils/src/zone.rs:363
12762024-12-21T01:15:37.074ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf file = illumos-utils/src/zone.rs:363
12772024-12-21T01:15:37.079ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4 file = illumos-utils/src/zone.rs:325
12782024-12-21T01:15:37.084ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21 file = illumos-utils/src/zone.rs:325
12792024-12-21T01:15:37.090ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d file = illumos-utils/src/zone.rs:325
12802024-12-21T01:15:37.096ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906 file = illumos-utils/src/zone.rs:325
12812024-12-21T01:15:37.101ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a file = illumos-utils/src/zone.rs:325
12822024-12-21T01:15:37.107ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4 file = illumos-utils/src/zone.rs:325
12832024-12-21T01:15:37.112ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39 file = illumos-utils/src/zone.rs:325
12842024-12-21T01:15:37.118ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4 file = illumos-utils/src/zone.rs:363
12852024-12-21T01:15:37.124ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21 file = illumos-utils/src/zone.rs:363
12862024-12-21T01:15:37.130ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d file = illumos-utils/src/zone.rs:363
12872024-12-21T01:15:37.136ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906 file = illumos-utils/src/zone.rs:363
12882024-12-21T01:15:37.143ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a file = illumos-utils/src/zone.rs:363
12892024-12-21T01:15:37.152ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4 file = illumos-utils/src/zone.rs:363
12902024-12-21T01:15:37.163ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39 file = illumos-utils/src/zone.rs:363
12912024-12-21T01:15:37.253ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12922024-12-21T01:15:37.309ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12932024-12-21T01:15:37.363ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12942024-12-21T01:15:37.422ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12952024-12-21T01:15:37.493ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
12962024-12-21T01:15:37.497ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
12972024-12-21T01:15:37.497ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
12982024-12-21T01:15:37.572ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte1", ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 186, 44]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
12992024-12-21T01:15:37.572ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = nexus
13002024-12-21T01:15:37.575ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
13012024-12-21T01:15:37.575ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
13022024-12-21T01:15:37.660ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte2", ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 131, 56]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
13032024-12-21T01:15:37.660ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = nexus
13042024-12-21T01:15:37.662ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
13052024-12-21T01:15:37.662ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
13062024-12-21T01:15:37.715ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte3", ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 216, 190]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
13072024-12-21T01:15:37.715ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = external_dns
13082024-12-21T01:15:37.717ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
13092024-12-21T01:15:37.717ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
13102024-12-21T01:15:37.768ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte4", ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 240, 244]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
13112024-12-21T01:15:37.769ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = external_dns
13122024-12-21T01:15:37.770ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
13132024-12-21T01:15:37.770ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
13142024-12-21T01:15:37.822ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte5", ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 234, 194]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
13152024-12-21T01:15:37.822ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1264 zone_type = nexus
13162024-12-21T01:15:37.822ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:83
13172024-12-21T01:15:37.822ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:90
13182024-12-21T01:15:37.822ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:83
13192024-12-21T01:15:37.822ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:90
13202024-12-21T01:15:37.822ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:83
13212024-12-21T01:15:37.822ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:90
13222024-12-21T01:15:37.822ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:83
13232024-12-21T01:15:37.822ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:90
13242024-12-21T01:15:37.825ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:118
13252024-12-21T01:15:37.826ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13262024-12-21T01:15:37.876ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:118
13272024-12-21T01:15:37.880ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:118
13282024-12-21T01:15:37.881ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13292024-12-21T01:15:37.883ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13302024-12-21T01:15:37.883ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:118
13312024-12-21T01:15:37.886ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13322024-12-21T01:15:37.886ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:83
13332024-12-21T01:15:37.886ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:90
13342024-12-21T01:15:38.078ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:118
13352024-12-21T01:15:38.081ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
13362024-12-21T01:15:38.081ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371 file = illumos-utils/src/zone.rs:325
13372024-12-21T01:15:38.101ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad file = illumos-utils/src/zone.rs:325
13382024-12-21T01:15:38.112ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490 file = illumos-utils/src/zone.rs:325
13392024-12-21T01:15:38.126ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153 file = illumos-utils/src/zone.rs:325
13402024-12-21T01:15:38.180ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371 file = illumos-utils/src/zone.rs:363
13412024-12-21T01:15:38.199ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad file = illumos-utils/src/zone.rs:363
13422024-12-21T01:15:38.210ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490 file = illumos-utils/src/zone.rs:363
13432024-12-21T01:15:38.231ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153 file = illumos-utils/src/zone.rs:363
13442024-12-21T01:15:38.243ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1 file = illumos-utils/src/zone.rs:325
13452024-12-21T01:15:38.285ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1 file = illumos-utils/src/zone.rs:363
13462024-12-21T01:15:46.386ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService18'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::15'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::15'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='77405b64-b17c-4def-80c2-2754095838a5'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13472024-12-21T01:15:46.388ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService22'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_e80b2509-2264-41c1-be0f-a3173a243a47/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1a'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='8c15f422-3d77-41aa-bb03-6325a0e37794'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13482024-12-21T01:15:46.389ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService23'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::16'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_10325d85-ce1b-49aa-8039-7e9482ec8803/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::16'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='e124f667-98d8-4632-8e42-cb6f2b3143ff'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13492024-12-21T01:15:46.390ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService14'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::14'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_7e02d0c4-4e5d-440d-94c1-593a67af3118/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::14'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='3a9b3385-684a-4838-b583-07fa54a2737d'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13502024-12-21T01:15:46.390ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService16'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::12'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::12'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='90fb0610-d5d7-4e1d-b3b8-a9e389f2e54f'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13512024-12-21T01:15:46.391ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService17'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::18'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_192cce10-69a8-4ed6-a459-9ca86a7206ec/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::18'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='5fd220c2-c69d-4559-995d-1d50fe011473'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13522024-12-21T01:15:46.391ZINFOSledAgent (ServiceManager): Booting oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
13532024-12-21T01:15:46.407ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService19'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::17'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::17'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='01edac9f-f5ba-4a8b-84cc-621da4543901'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13542024-12-21T01:15:46.408ZINFOSledAgent (ServiceManager): Booting oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
13552024-12-21T01:15:46.416ZINFOSledAgent (ServiceManager): Booting oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
13562024-12-21T01:15:46.426ZINFOSledAgent (ServiceManager): Booting oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
13572024-12-21T01:15:46.435ZINFOSledAgent (ServiceManager): Booting oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
13582024-12-21T01:15:46.444ZINFOSledAgent (ServiceManager): Booting oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
13592024-12-21T01:15:46.455ZINFOSledAgent (ServiceManager): Booting oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
13602024-12-21T01:15:46.717ZINFOSledAgent (ServiceManager): Profile for oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService10'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::d'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/oximeter"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="id" value='639157e9-2713-43a8-90e8-75860cd77732'/> <propval type="astring" name="address" value='[fd00:1122:3344:101::d]:12223'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13612024-12-21T01:15:46.718ZINFOSledAgent (ServiceManager): Booting oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
13622024-12-21T01:15:46.961ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 67.21474ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
13632024-12-21T01:15:46.974ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 73.574054ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
13642024-12-21T01:15:46.986ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 68.084545ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
13652024-12-21T01:15:47.001ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 42.426312ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
13662024-12-21T01:15:47.016ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 48.548219ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
13672024-12-21T01:15:47.030ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 64.301127ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
13682024-12-21T01:15:47.044ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 63.664924ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
13692024-12-21T01:15:47.061ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 77.832572ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
13702024-12-21T01:15:47.126ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 77.348942ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
13712024-12-21T01:15:47.148ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 99.13693ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
13722024-12-21T01:15:47.222ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 84.034845ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
13732024-12-21T01:15:47.249ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 144.636819ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
13742024-12-21T01:15:47.266ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 80.527207ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
13752024-12-21T01:15:47.326ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 106.090476ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
13762024-12-21T01:15:47.349ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 68.226633ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
13772024-12-21T01:15:47.372ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 211.964876ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
13782024-12-21T01:15:47.407ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 193.773803ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
13792024-12-21T01:15:47.423ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 232.210215ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
13802024-12-21T01:15:47.438ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 238.817795ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
13812024-12-21T01:15:47.451ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 217.551123ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
13822024-12-21T01:15:47.464ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 241.025516ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
13832024-12-21T01:15:47.477ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 127.111413ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
13842024-12-21T01:15:47.489ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 109.743409ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
13852024-12-21T01:15:47.595ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 359.98392ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
13862024-12-21T01:15:47.612ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 189.952614ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
13872024-12-21T01:15:47.622ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 256.082129ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
13882024-12-21T01:15:47.632ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 363.87355ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
13892024-12-21T01:15:47.667ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 204.485136ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
13902024-12-21T01:15:47.682ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 548.144378ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
13912024-12-21T01:15:47.694ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 542.502305ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
13922024-12-21T01:15:47.718ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 526.154145ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
13932024-12-21T01:15:47.815ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 422.754128ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
13942024-12-21T01:15:47.842ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService26'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::8'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte3'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::8]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.5:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13952024-12-21T01:15:47.843ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
13962024-12-21T01:15:47.885ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 438.302978ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
13972024-12-21T01:15:47.896ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 796.686151ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
13982024-12-21T01:15:47.970ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 438.755994ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
13992024-12-21T01:15:48.101ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 433.231955ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
14002024-12-21T01:15:48.275ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 734.289565ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
14012024-12-21T01:15:48.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 415.997269ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
14022024-12-21T01:15:48.465ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 672.564405ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
14032024-12-21T01:15:48.570ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 1.037653594s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
14042024-12-21T01:15:48.628ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 1.251665043s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
14052024-12-21T01:15:48.673ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 545.300829ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
14062024-12-21T01:15:48.757ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 858.431156ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
14072024-12-21T01:15:48.803ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 1.161480438s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
14082024-12-21T01:15:48.836ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 854.820129ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
14092024-12-21T01:15:48.855ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 51.584519ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
14102024-12-21T01:15:48.927ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 130.349963ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
14112024-12-21T01:15:49.027ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 850.902889ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
14122024-12-21T01:15:49.028ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService12'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::10'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::10'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14132024-12-21T01:15:49.028ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService15'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::11'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::11'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14142024-12-21T01:15:49.028ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
14152024-12-21T01:15:49.038ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
14162024-12-21T01:15:49.049ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService11'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::f'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::f'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14172024-12-21T01:15:49.050ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
14182024-12-21T01:15:49.083ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 290.058604ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
14192024-12-21T01:15:49.123ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService13'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::13'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_41cc83ed-adbe-492f-ae31-7074b31692b0/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::13'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='2d29a3a1-1252-49e6-bedd-cea7fa3ab65c'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14202024-12-21T01:15:49.123ZINFOSledAgent (ServiceManager): Booting oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
14212024-12-21T01:15:49.132ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService21'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::19'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_34789575-52b6-411b-9614-90f11629cb04/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::19'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='c710fc76-b936-46ca-80bc-0a24a51039c9'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14222024-12-21T01:15:49.132ZINFOSledAgent (ServiceManager): Booting oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
14232024-12-21T01:15:49.157ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 1.167128826s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
14242024-12-21T01:15:49.262ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 1.19089163s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
14252024-12-21T01:15:49.343ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 58.373755ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
14262024-12-21T01:15:49.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 37.538783ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
14272024-12-21T01:15:49.368ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 51.9692ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
14282024-12-21T01:15:49.388ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 234.296359ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
14292024-12-21T01:15:49.403ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 142.984461ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
14302024-12-21T01:15:49.413ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 100.136756ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
14312024-12-21T01:15:49.423ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 71.175588ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
14322024-12-21T01:15:49.446ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 61.391116ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
14332024-12-21T01:15:49.456ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 59.751233ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
14342024-12-21T01:15:49.550ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 96.475176ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
14352024-12-21T01:15:49.621ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 180.640713ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
14362024-12-21T01:15:49.670ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 217.595683ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
14372024-12-21T01:15:49.685ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 114.172016ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
14382024-12-21T01:15:49.701ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 207.058157ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
14392024-12-21T01:15:49.725ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 555.995267ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
14402024-12-21T01:15:49.742ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 998.708291ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
14412024-12-21T01:15:49.757ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 151.979493ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
14422024-12-21T01:15:49.773ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 1.454318604s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
14432024-12-21T01:15:49.787ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 1.404540232s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
14442024-12-21T01:15:49.816ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 123.663992ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
14452024-12-21T01:15:49.832ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 398.54225ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
14462024-12-21T01:15:49.843ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService24'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::c'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte1'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14472024-12-21T01:15:49.844ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService25'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte2'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14482024-12-21T01:15:49.844ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService28'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte5'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14492024-12-21T01:15:49.844ZINFOSledAgent (ServiceManager): Booting oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
14502024-12-21T01:15:49.855ZINFOSledAgent (ServiceManager): Booting oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
14512024-12-21T01:15:49.864ZINFOSledAgent (ServiceManager): Booting oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
14522024-12-21T01:15:49.874ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService27'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::9'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte4'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::9]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.6:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14532024-12-21T01:15:49.874ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
14542024-12-21T01:15:49.895ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 1.454360939s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
14552024-12-21T01:15:49.909ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 570.821854ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
14562024-12-21T01:15:49.922ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 937.656849ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
14572024-12-21T01:15:49.938ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 530.637962ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
14582024-12-21T01:15:49.972ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 343.658649ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
14592024-12-21T01:15:50.017ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 1.243638772s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
14602024-12-21T01:15:50.045ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 356.294217ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
14612024-12-21T01:15:50.160ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 70.314803ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
14622024-12-21T01:15:50.193ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 64.03196ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
14632024-12-21T01:15:50.206ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 44.30471ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
14642024-12-21T01:15:50.217ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 42.106552ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
14652024-12-21T01:15:50.244ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 129.245801ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
14662024-12-21T01:15:50.258ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 494.077998ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
14672024-12-21T01:15:50.270ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 70.1125ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
14682024-12-21T01:15:50.282ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 97.724636ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
14692024-12-21T01:15:50.296ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 1.491153262s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
14702024-12-21T01:15:50.309ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 57.576497ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
14712024-12-21T01:15:50.325ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 548.579616ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
14722024-12-21T01:15:50.335ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 784.608641ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
14732024-12-21T01:15:50.355ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 251.547791ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
14742024-12-21T01:15:50.378ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 244.073366ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
14752024-12-21T01:15:50.389ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 206.099974ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
14762024-12-21T01:15:50.399ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 203.036887ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
14772024-12-21T01:15:50.414ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 814.384779ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
14782024-12-21T01:15:50.469ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 1.095040218s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
14792024-12-21T01:15:50.563ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 1.191948623s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
14802024-12-21T01:15:50.636ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 1.199227377s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
14812024-12-21T01:15:50.680ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 595.898504ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
14822024-12-21T01:15:50.719ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 578.009614ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
14832024-12-21T01:15:50.747ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 295.393945ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
14842024-12-21T01:15:50.766ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 433.512327ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
14852024-12-21T01:15:50.781ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 717.664553ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
14862024-12-21T01:15:50.796ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 1.475322916s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
14872024-12-21T01:15:50.876ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 1.28601752s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
14882024-12-21T01:15:50.891ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 783.026728ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
14892024-12-21T01:15:51.061ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 613.514738ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
14902024-12-21T01:15:51.137ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 1.196902443s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
14912024-12-21T01:15:51.210ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 1.001253617s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
14922024-12-21T01:15:51.229ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 659.681217ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
14932024-12-21T01:15:51.246ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 1.181785386s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
14942024-12-21T01:15:51.263ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 887.560718ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
14952024-12-21T01:15:51.282ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 1.012015453s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
14962024-12-21T01:15:51.306ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 633.699108ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
14972024-12-21T01:15:51.323ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 472.973004ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
14982024-12-21T01:15:51.364ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 813.519439ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
14992024-12-21T01:15:51.513ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 561.517839ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
15002024-12-21T01:15:51.578ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 615.017086ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
15012024-12-21T01:15:51.689ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 687.219422ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
15022024-12-21T01:15:51.705ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 1.386916402s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
15032024-12-21T01:15:51.772ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 1.276420015s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
15042024-12-21T01:15:51.804ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 823.425431ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
15052024-12-21T01:15:51.821ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 784.526262ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
15062024-12-21T01:15:51.852ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 902.468845ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
15072024-12-21T01:15:51.906ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 1.285749645s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
15082024-12-21T01:15:51.956ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 591.352119ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
15092024-12-21T01:15:52.091ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 961.96542ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
15102024-12-21T01:15:52.165ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 914.05299ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
15112024-12-21T01:15:52.178ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 986.403795ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
15122024-12-21T01:15:52.190ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 1.173558588s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
15132024-12-21T01:15:52.207ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 869.898216ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
15142024-12-21T01:15:52.223ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 1.313646756s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
15152024-12-21T01:15:52.286ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 565.497566ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
15162024-12-21T01:15:52.307ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 692.012712ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
15172024-12-21T01:15:52.349ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 516.133951ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
15182024-12-21T01:15:52.390ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 1.474182334s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
15192024-12-21T01:15:52.441ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 834.464203ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
15202024-12-21T01:15:52.562ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 1.017217654s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
15212024-12-21T01:15:52.619ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 1.448283548s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
15222024-12-21T01:15:52.641ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 1.130209638s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
15232024-12-21T01:15:52.989ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 2.824241938s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
15242024-12-21T01:15:53.003ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 1.006454646s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
15252024-12-21T01:15:53.016ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 842.613325ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
15262024-12-21T01:15:53.029ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 907.848962ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
15272024-12-21T01:15:53.041ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 725.292082ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
15282024-12-21T01:15:53.061ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 1.374151721s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
15292024-12-21T01:15:53.074ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 863.756819ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
15302024-12-21T01:15:53.090ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 544.981845ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
15312024-12-21T01:15:53.099ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 1.391025838s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
15322024-12-21T01:15:53.132ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService20'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::e'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::e'/> <propval type="astring" name="listen_port" value='8123'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse-admin-single"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::e]:8888'/> <propval type="astring" name="ch_address" value='[fd00:1122:3344:101::e]:9000'/> <propval type="astring" name="ch_binary" value='/opt/oxide/clickhouse/clickhouse'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
15332024-12-21T01:15:53.139ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 777.027376ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
15342024-12-21T01:15:53.139ZINFOSledAgent (ServiceManager): Booting oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
15352024-12-21T01:15:53.177ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 1.438334827s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
15362024-12-21T01:15:53.204ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 1.051514878s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
15372024-12-21T01:15:53.286ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 702.702621ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
15382024-12-21T01:15:53.367ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 73.873367ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
15392024-12-21T01:15:53.375ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 761.556717ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
15402024-12-21T01:15:53.449ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 53.04059ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
15412024-12-21T01:15:53.511ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 101.193562ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
15422024-12-21T01:15:53.546ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 1.107599796s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
15432024-12-21T01:15:53.588ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 1.135529734s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
15442024-12-21T01:15:53.620ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 468.23743ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
15452024-12-21T01:15:53.644ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 540.980116ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
15462024-12-21T01:15:53.776ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 532.215137ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
15472024-12-21T01:15:53.784ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 851.710093ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
15482024-12-21T01:15:53.868ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 655.353008ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
15492024-12-21T01:15:53.877ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 721.78062ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
15502024-12-21T01:15:53.924ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 1.13453966s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
15512024-12-21T01:15:53.945ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 644.446523ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
15522024-12-21T01:15:53.954ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 1.293210929s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
15532024-12-21T01:15:53.998ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 1.396288678s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
15542024-12-21T01:15:54.019ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 1.305980308s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
15552024-12-21T01:15:54.076ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 524.548018ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
15562024-12-21T01:15:54.097ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 495.614024ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
15572024-12-21T01:15:54.146ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 900.206562ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
15582024-12-21T01:15:54.195ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 1.099783033s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
15592024-12-21T01:15:54.266ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 1.124382274s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
15602024-12-21T01:15:54.318ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 870.181032ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
15612024-12-21T01:15:54.448ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 1.25061421s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
15622024-12-21T01:15:54.505ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 525.208922ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
15632024-12-21T01:15:54.533ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 526.167887ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
15642024-12-21T01:15:54.599ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 629.667861ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
15652024-12-21T01:15:54.607ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 928.205682ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
15662024-12-21T01:15:54.614ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 930.604328ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
15672024-12-21T01:15:54.622ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 1.448069891s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
15682024-12-21T01:15:54.629ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 1.256584322s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
15692024-12-21T01:15:54.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 592.953419ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
15702024-12-21T01:15:54.662ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 1.162525879s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
15712024-12-21T01:15:54.733ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 709.926889ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
15722024-12-21T01:15:55.041ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 939.071118ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
15732024-12-21T01:15:55.056ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 1.344923576s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
15742024-12-21T01:15:55.068ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 1.36899561s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
15752024-12-21T01:15:55.086ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 671.785904ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
15762024-12-21T01:15:55.198ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 1.047930392s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
15772024-12-21T01:15:55.239ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 772.92133ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
15782024-12-21T01:15:55.248ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 1.092085333s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
15792024-12-21T01:15:55.256ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 782.306302ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
15802024-12-21T01:15:55.303ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 1.353412669s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
15812024-12-21T01:15:55.334ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 1.0133912s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
15822024-12-21T01:15:55.399ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 997.369515ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
15832024-12-21T01:15:55.407ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 1.402684785s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
15842024-12-21T01:15:55.454ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 1.167353289s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
15852024-12-21T01:15:55.543ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 1.134445028s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
15862024-12-21T01:15:55.553ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 830.993113ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
15872024-12-21T01:15:55.708ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 1.212521135s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
15882024-12-21T01:15:55.766ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 531.085427ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
15892024-12-21T01:15:56.029ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 840.057804ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
15902024-12-21T01:15:56.038ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 717.233271ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
15912024-12-21T01:15:56.046ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 586.223334ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
15922024-12-21T01:15:56.055ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 1.40664127s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
15932024-12-21T01:15:56.062ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 864.587287ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
15942024-12-21T01:15:56.079ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 1.450921482s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
15952024-12-21T01:15:56.256ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 1.163765898s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
15962024-12-21T01:15:56.307ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 664.495309ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
15972024-12-21T01:15:56.350ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 1.404283943s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
15982024-12-21T01:15:56.358ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 930.018213ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
15992024-12-21T01:15:56.394ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 1.160103504s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
16002024-12-21T01:15:56.407ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 907.596261ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
16012024-12-21T01:15:56.417ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 1.18176431s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
16022024-12-21T01:15:56.448ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 1.345277716s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
16032024-12-21T01:15:56.630ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 780.807962ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
16042024-12-21T01:15:56.640ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 824.539727ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
16052024-12-21T01:15:56.665ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 1.07163236s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
16062024-12-21T01:15:56.685ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 879.811109ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
16072024-12-21T01:15:56.771ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 1.40460454s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
16082024-12-21T01:15:56.824ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 1.449257915s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
16092024-12-21T01:15:56.880ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 1.270662819s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
16102024-12-21T01:15:56.931ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 627.725432ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
16112024-12-21T01:15:56.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 897.524116ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
16122024-12-21T01:15:56.981ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 1.408172068s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
16132024-12-21T01:15:57.299ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 1.100546399s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
16142024-12-21T01:15:57.324ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 1.100773601s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
16152024-12-21T01:15:57.421ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 965.430146ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
16162024-12-21T01:15:57.430ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 825.999425ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
16172024-12-21T01:15:57.472ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 1.135325676s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
16182024-12-21T01:15:57.482ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 1.461831527s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
16192024-12-21T01:15:57.541ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 1.372122639s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
16202024-12-21T01:15:57.564ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 1.361671063s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
16212024-12-21T01:15:57.573ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 1.101770567s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
16222024-12-21T01:15:57.581ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 947.58525ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
16232024-12-21T01:15:57.608ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 772.272299ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
16242024-12-21T01:15:57.747ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 633.600885ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
16252024-12-21T01:15:57.765ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 1.366201458s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
16262024-12-21T01:15:57.803ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 1.133411187s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
16272024-12-21T01:15:57.848ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 639.233662ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
16282024-12-21T01:15:58.161ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 772.59769ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
16292024-12-21T01:15:58.188ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 704.311508ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
16302024-12-21T01:15:58.266ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 709.043731ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
16312024-12-21T01:15:58.284ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 1.157426856s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
16322024-12-21T01:15:58.390ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 524.622515ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
16332024-12-21T01:15:58.400ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 595.651957ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
16342024-12-21T01:15:58.408ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 1.018769776s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
16352024-12-21T01:15:58.418ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 659.388329ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
16362024-12-21T01:15:58.425ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 1.283301771s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
16372024-12-21T01:15:58.435ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 1.468057696s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
16382024-12-21T01:15:58.498ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 532.926133ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
16392024-12-21T01:15:58.539ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 843.194758ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
16402024-12-21T01:15:58.619ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 860.465457ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
16412024-12-21T01:15:58.688ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 633.921093ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
16422024-12-21T01:15:58.903ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 503.605597ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
16432024-12-21T01:15:58.922ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 1.101121644s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
16442024-12-21T01:15:58.932ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 867.839992ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
16452024-12-21T01:15:58.941ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 674.009939ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
16462024-12-21T01:15:58.950ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 1.174777461s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
16472024-12-21T01:15:58.960ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 726.819893ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
16482024-12-21T01:15:58.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 1.376779483s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
16492024-12-21T01:15:58.985ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 846.628534ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
16502024-12-21T01:15:59.006ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 1.271509049s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
16512024-12-21T01:15:59.042ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 989.720476ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
16522024-12-21T01:15:59.088ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 1.26253292s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
16532024-12-21T01:15:59.141ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 688.350717ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
16542024-12-21T01:15:59.332ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 785.781119ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
16552024-12-21T01:15:59.393ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 941.054059ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
16562024-12-21T01:15:59.418ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 1.496672148s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
16572024-12-21T01:15:59.439ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 590.536907ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
16582024-12-21T01:15:59.451ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 502.054209ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
16592024-12-21T01:15:59.491ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 1.142297102s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
16602024-12-21T01:15:59.626ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 1.14525126s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
16612024-12-21T01:15:59.696ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 1.382294447s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
16622024-12-21T01:15:59.718ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 594.499146ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
16632024-12-21T01:15:59.809ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 1.49750453s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
16642024-12-21T01:15:59.840ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 962.129499ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
16652024-12-21T01:15:59.850ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 1.392668716s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
16662024-12-21T01:15:59.913ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 969.52304ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
16672024-12-21T01:15:59.964ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 877.618178ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
16682024-12-21T01:16:00.035ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 1.004529291s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
16692024-12-21T01:16:00.043ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 756.537187ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
16702024-12-21T01:16:00.053ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 785.261333ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
16712024-12-21T01:16:00.129ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 1.171475021s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
16722024-12-21T01:16:00.139ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 1.496303721s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
16732024-12-21T01:16:00.289ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 1.07845351s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
16742024-12-21T01:16:00.323ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 694.694966ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
16752024-12-21T01:16:00.344ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 1.049101297s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
16762024-12-21T01:16:00.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 1.120115821s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
16772024-12-21T01:16:00.365ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 521.382446ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
16782024-12-21T01:16:00.643ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 1.489287745s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
16792024-12-21T01:16:00.783ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 1.386091688s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
16802024-12-21T01:16:00.811ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 911.6702ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
16812024-12-21T01:16:00.819ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 1.092762722s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
16822024-12-21T01:16:00.849ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 522.031235ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
16832024-12-21T01:16:00.859ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 597.983541ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
16842024-12-21T01:16:00.893ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 820.591906ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
16852024-12-21T01:16:00.902ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 618.915173ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
16862024-12-21T01:16:00.928ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 751.131036ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
16872024-12-21T01:16:01.030ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 1.069306255s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
16882024-12-21T01:16:01.063ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 1.355387449s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
16892024-12-21T01:16:01.089ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 869.44123ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
16902024-12-21T01:16:01.253ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 1.216623715s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
16912024-12-21T01:16:01.311ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 1.337011312s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
16922024-12-21T01:16:01.320ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 1.443987211s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
16932024-12-21T01:16:01.378ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 560.011931ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
16942024-12-21T01:16:01.386ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 1.184122145s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
16952024-12-21T01:16:01.403ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 1.119154834s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
16962024-12-21T01:16:01.472ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 816.40429ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
16972024-12-21T01:16:01.488ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 681.841722ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
16982024-12-21T01:16:01.537ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 1.247768768s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
16992024-12-21T01:16:01.650ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 1.173239022s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
17002024-12-21T01:16:01.691ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 1.113781607s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
17012024-12-21T01:16:01.725ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 1.399990483s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
17022024-12-21T01:16:01.737ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 729.429996ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
17032024-12-21T01:16:01.926ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 914.128977ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
17042024-12-21T01:16:01.949ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 643.905936ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
17052024-12-21T01:16:01.973ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 1.330342829s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
17062024-12-21T01:16:02.111ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 903.736947ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
17072024-12-21T01:16:02.147ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 1.005658103s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
17082024-12-21T01:16:02.185ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 897.79973ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
17092024-12-21T01:16:02.201ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 679.851331ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
17102024-12-21T01:16:02.303ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 1.377485379s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
17112024-12-21T01:16:02.432ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 1.479981156s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
17122024-12-21T01:16:02.479ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 1.343911026s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
17132024-12-21T01:16:02.491ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 1.384259209s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
17142024-12-21T01:16:02.533ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 650.875175ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
17152024-12-21T01:16:02.585ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 1.273244851s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
17162024-12-21T01:16:02.612ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 831.975834ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
17172024-12-21T01:16:02.666ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 653.816437ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
17182024-12-21T01:16:02.781ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 864.541232ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
17192024-12-21T01:16:02.800ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 1.413501327s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
17202024-12-21T01:16:02.818ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 548.752841ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
17212024-12-21T01:16:02.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 1.484862101s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
17222024-12-21T01:16:02.856ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 698.946095ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
17232024-12-21T01:16:02.898ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 1.035026426s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
17242024-12-21T01:16:03.030ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 1.1953765s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
17252024-12-21T01:16:03.101ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 652.718402ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
17262024-12-21T01:16:03.141ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 745.055417ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
17272024-12-21T01:16:03.169ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39") failed: Property not found. retry in 1.383730429s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
17282024-12-21T01:16:03.201ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 1.301657729s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
17292024-12-21T01:16:03.325ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 1.252316225s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
17302024-12-21T01:16:03.342ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 732.443142ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
17312024-12-21T01:16:03.381ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 707.28635ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
17322024-12-21T01:16:03.466ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 811.021152ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
17332024-12-21T01:16:03.574ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 1.009565774s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
17342024-12-21T01:16:03.662ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 1.137616836s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
17352024-12-21T01:16:03.697ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4") failed: Property not found. retry in 1.109606748s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
17362024-12-21T01:16:03.768ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 1.422911994s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
17372024-12-21T01:16:03.837ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 890.851241ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
17382024-12-21T01:16:03.872ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf") failed: Property not found. retry in 1.139391585s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
17392024-12-21T01:16:03.890ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732") failed: Property not found. retry in 1.393383027s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
17402024-12-21T01:16:03.902ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 582.289064ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
17412024-12-21T01:16:03.924ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 1.125463393s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
17422024-12-21T01:16:03.952ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 964.869052ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
17432024-12-21T01:16:04.094ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 848.718782ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
17442024-12-21T01:16:04.112ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 666.246556ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
17452024-12-21T01:16:04.232ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4") failed: Property not found. retry in 1.102228279s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
17462024-12-21T01:16:04.249ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 1.325114113s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
17472024-12-21T01:16:04.293ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 1.339619587s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
17482024-12-21T01:16:04.337ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 1.310047276s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
17492024-12-21T01:16:04.496ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 1.109768714s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
17502024-12-21T01:16:04.515ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 827.250119ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
17512024-12-21T01:16:04.588ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 1.196496656s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
17522024-12-21T01:16:04.602ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 722.845622ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
17532024-12-21T01:16:04.749ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 1.044022396s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
17542024-12-21T01:16:04.794ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4") failed: Property not found. retry in 919.571252ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
17552024-12-21T01:16:04.820ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d") failed: Property not found. retry in 1.385014549s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
17562024-12-21T01:16:04.935ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21") failed: Property not found. retry in 840.131187ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
17572024-12-21T01:16:04.956ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 1.165383888s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
17582024-12-21T01:16:05.066ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 1.487162455s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
17592024-12-21T01:16:05.204ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 1.077941673s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
17602024-12-21T01:16:05.342ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 909.611826ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
17612024-12-21T01:16:05.373ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 646.270551ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
17622024-12-21T01:16:05.587ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 691.318841ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
17632024-12-21T01:16:05.618ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 1.287356421s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
17642024-12-21T01:16:05.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 865.440769ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
17652024-12-21T01:16:05.659ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 1.282811472s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
17662024-12-21T01:16:05.814ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc") failed: Property not found. retry in 1.265482416s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
17672024-12-21T01:16:05.828ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 1.143573701s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
17682024-12-21T01:16:06.034ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 1.313614984s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
17692024-12-21T01:16:06.134ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 1.433762086s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
17702024-12-21T01:16:06.264ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490") failed: Property not found. retry in 824.925155ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
17712024-12-21T01:16:06.291ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0") failed: Property not found. retry in 1.310166263s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
17722024-12-21T01:16:06.304ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 987.800125ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
17732024-12-21T01:16:06.322ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:41933
17742024-12-21T01:16:06.327ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 502 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:41933 req_id = 67d52b9e-51aa-4f94-9c1c-efc859b3d44c response_code = 200 uri = /baseboard
17752024-12-21T01:16:06.525ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 1.035400722s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
17762024-12-21T01:16:06.564ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153") failed: Property not found. retry in 1.292835144s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
17772024-12-21T01:16:06.917ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 1.098294362s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
17782024-12-21T01:16:06.952ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 1.141718014s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
17792024-12-21T01:16:06.982ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a") failed: Property not found. retry in 1.267150227s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
17802024-12-21T01:16:07.302ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 900.03816ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
17812024-12-21T01:16:07.357ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 728.638555ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
17822024-12-21T01:16:07.570ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 574.552189ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
17832024-12-21T01:16:07.578ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a") failed: Property not found. retry in 522.479821ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
17842024-12-21T01:16:08.026ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 760.282577ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
17852024-12-21T01:16:08.095ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 755.224791ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
17862024-12-21T01:16:08.103ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 661.100319ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
17872024-12-21T01:16:08.154ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 1.160548924s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
17882024-12-21T01:16:08.211ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 699.576131ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
17892024-12-21T01:16:08.774ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 1.493512756s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
17902024-12-21T01:16:08.797ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1") failed: Property not found. retry in 855.845701ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
17912024-12-21T01:16:08.860ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 582.903306ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
17922024-12-21T01:16:08.919ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 1.257557301s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
17932024-12-21T01:16:09.325ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 1.39883441s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
17942024-12-21T01:16:09.455ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 1.12228548s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
17952024-12-21T01:16:10.187ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad") failed: Property not found. retry in 1.27443138s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
17962024-12-21T01:16:10.276ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1") failed: Property not found. retry in 812.482089ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
17972024-12-21T01:16:10.587ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 1.452656602s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
17982024-12-21T01:16:10.735ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371") failed: Property not found. retry in 546.787462ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
17992024-12-21T01:16:11.022ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 4.726434942s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
18002024-12-21T01:16:12.051ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 1.319651186s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
18012024-12-21T01:16:13.381ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 731.115436ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
18022024-12-21T01:16:14.122ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 1.35553217s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
18032024-12-21T01:16:15.488ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 581.520647ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
18042024-12-21T01:16:15.791ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 6.351768534s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
18052024-12-21T01:16:16.081ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 895.487125ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
18062024-12-21T01:16:16.987ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 888.659568ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
18072024-12-21T01:16:17.886ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 580.425205ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
18082024-12-21T01:16:18.478ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 868.874321ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
18092024-12-21T01:16:19.357ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906") failed: Property not found. retry in 1.471496333s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
18102024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_nexus_fe9c2ca3-0266-4081-952b-4cb3f2bf1371
18112024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_nexus_ee35abf0-6140-47bd-8a0f-da60e1b0cdad
18122024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_external_dns_24428c69-128b-4995-bc7a-addc66912490
18132024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_oximeter_639157e9-2713-43a8-90e8-75860cd77732
18142024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_pantry_6e0a9baf-a85f-4912-82e5-6107854998fc
18152024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_pantry_1714c239-c2ea-4fdc-a2ff-2fdc548654f0
18162024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_698a7a9e-913f-4d1d-8acf-e625ceef34b1
18172024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_1519a09c-6c94-4612-b980-ea6d97dcd5c4
18182024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_pantry_c4f5f6e5-b38f-4a7c-a95b-ea2551b15b8a
18192024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_dad237e0-cce5-47d0-a98d-2d417f8b60cf
18202024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_b3847803-729f-4a75-bd01-c44489c8d3e4
18212024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_external_dns_c4364b0f-0b76-413b-9126-391216a47153
18222024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_b387a3bb-a500-4bf1-a8ce-4c3de63bfe21
18232024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_6c21be66-efc8-42a3-af47-0191d8ec508d
18242024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_nexus_bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1
18252024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_clickhouse_643661ea-2572-4236-b774-49e58b17e906
18262024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_1f9393ec-0459-42a3-ab95-22a9c9c1839a
18272024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_94b13f11-1aeb-47d6-9e22-e77f14ed68f4
18282024-12-21T01:16:20.863ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3325 zone = oxz_crucible_24749034-6e5c-4b03-9d90-11debee9bb39
18292024-12-21T01:16:20.864ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
18302024-12-21T01:16:20.868ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
18312024-12-21T01:16:20.878ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44641568 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:65408 req_id = ba9ee60c-20a3-4c7b-b675-50d919f4aeaa response_code = 204 uri = /omicron-zones
18322024-12-21T01:16:20.887ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1415
18332024-12-21T01:16:20.893ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:790
18342024-12-21T01:16:20.894ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::b]:12221 file = sled-agent/src/rack_setup/service.rs:802
18352024-12-21T01:16:20.927ZINFOSledAgent (RSS): rack_network_config: RackNetworkConfigV2 { bfd: [], bgp: [], infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], autoneg: false, bgp_peers: [], lldp: None, port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, rib_priority: None, vlan_id: None, }, ], switch: Switch0, tx_eq: None, uplink_port_fec: Some( None, ), uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, } file = sled-agent/src/rack_setup/service.rs:1016
18362024-12-21T01:16:21.180ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "4f926e1b-7de1-4d8a-84b8-56542cee7b75", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:21 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"18f9683c-e647-4a58-a294-698fb36fa534\"", request_id: "4f926e1b-7de1-4d8a-84b8-56542cee7b75" } file = sled-agent/src/rack_setup/service.rs:1078
18372024-12-21T01:16:21.315ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "ab6a07d0-e3d6-4481-8621-6df6542a0fe7", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:20 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"18f9683c-e647-4a58-a294-698fb36fa534\"", request_id: "ab6a07d0-e3d6-4481-8621-6df6542a0fe7" } file = sled-agent/src/rack_setup/service.rs:1078
18382024-12-21T01:16:21.422ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "b3187e68-cff9-4c9f-9903-ed45617bbdc4", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:21 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"18f9683c-e647-4a58-a294-698fb36fa534\"", request_id: "b3187e68-cff9-4c9f-9903-ed45617bbdc4" } file = sled-agent/src/rack_setup/service.rs:1078
18392024-12-21T01:16:21.644ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "41bb166c-eb36-4601-a6c6-b167f6f47bd6", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:21 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"18f9683c-e647-4a58-a294-698fb36fa534\"", request_id: "41bb166c-eb36-4601-a6c6-b167f6f47bd6" } file = sled-agent/src/rack_setup/service.rs:1078
18402024-12-21T01:16:21.761ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "c4c3cb92-af94-405e-bec0-090f67d6a4ba", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:21 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"18f9683c-e647-4a58-a294-698fb36fa534\"", request_id: "c4c3cb92-af94-405e-bec0-090f67d6a4ba" } file = sled-agent/src/rack_setup/service.rs:1078
18412024-12-21T01:16:21.914ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "593248cb-2e74-4843-84b3-3fc79da4017a", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:21 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: rack with id \"18f9683c-e647-4a58-a294-698fb36fa534\"", request_id: "593248cb-2e74-4843-84b3-3fc79da4017a" } file = sled-agent/src/rack_setup/service.rs:1078
18422024-12-21T01:16:22.376ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 15.256088339s error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"b27003c4-e5de-40a3-aba1-c1c74e60aa7d\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Sat, 21 Dec 2024 01:16:22 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"b27003c4-e5de-40a3-aba1-c1c74e60aa7d\\" }" file = oximeter/producer/src/lib.rs:420 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
18432024-12-21T01:16:23.572ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "bcc73c1e-e641-4629-871e-2f64a12e9039", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:23 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"b1eebc13-191d-4a5d-a35e-84ae64fecac9\"", request_id: "bcc73c1e-e641-4629-871e-2f64a12e9039" } file = sled-agent/src/rack_setup/service.rs:1078
18442024-12-21T01:16:24.527ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "f0e8a14f-f0ff-4256-a81b-b9c294db59b8", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:24 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"b1eebc13-191d-4a5d-a35e-84ae64fecac9\"", request_id: "f0e8a14f-f0ff-4256-a81b-b9c294db59b8" } file = sled-agent/src/rack_setup/service.rs:1078
18452024-12-21T01:16:25.158ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "31660cca-6e4a-44f0-b01a-69b994c6021a", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:25 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"b1eebc13-191d-4a5d-a35e-84ae64fecac9\"", request_id: "31660cca-6e4a-44f0-b01a-69b994c6021a" } file = sled-agent/src/rack_setup/service.rs:1078
18462024-12-21T01:16:25.674ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "840a8fc5-ba8c-4491-9bf5-c01e6e84ece6", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:25 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"b1eebc13-191d-4a5d-a35e-84ae64fecac9\"", request_id: "840a8fc5-ba8c-4491-9bf5-c01e6e84ece6" } file = sled-agent/src/rack_setup/service.rs:1078
18472024-12-21T01:16:26.625ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "7e4670be-01c5-4769-a731-28a5eba1f95d", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:26 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"b1eebc13-191d-4a5d-a35e-84ae64fecac9\"", request_id: "7e4670be-01c5-4769-a731-28a5eba1f95d" } file = sled-agent/src/rack_setup/service.rs:1078
18482024-12-21T01:16:27.661ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "66649763-f316-451d-8dd0-dd21e0fbe8ca", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:27 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"b1eebc13-191d-4a5d-a35e-84ae64fecac9\"", request_id: "66649763-f316-451d-8dd0-dd21e0fbe8ca" } file = sled-agent/src/rack_setup/service.rs:1078
18492024-12-21T01:16:28.482ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "3d694663-993d-4d98-bb50-81d1e00e3ea9", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:28 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"b1eebc13-191d-4a5d-a35e-84ae64fecac9\"", request_id: "3d694663-993d-4d98-bb50-81d1e00e3ea9" } file = sled-agent/src/rack_setup/service.rs:1078
18502024-12-21T01:16:30.173ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "dc79f626-a863-452b-b265-eb646d2c845f", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:29 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"b1eebc13-191d-4a5d-a35e-84ae64fecac9\"", request_id: "dc79f626-a863-452b-b265-eb646d2c845f" } file = sled-agent/src/rack_setup/service.rs:1078
18512024-12-21T01:16:31.302ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "9daffe21-5dd0-4e20-8154-61f3516701f5", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:31 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"b1eebc13-191d-4a5d-a35e-84ae64fecac9\"", request_id: "9daffe21-5dd0-4e20-8154-61f3516701f5" } file = sled-agent/src/rack_setup/service.rs:1078
18522024-12-21T01:16:32.240ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "da63e670-7ecd-4465-a0d7-24cf7869e3f9", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:32 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"b1eebc13-191d-4a5d-a35e-84ae64fecac9\"", request_id: "da63e670-7ecd-4465-a0d7-24cf7869e3f9" } file = sled-agent/src/rack_setup/service.rs:1078
18532024-12-21T01:16:33.391ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "9135bb66-12f7-485a-9e74-46e5276b44e7", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:33 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"b1eebc13-191d-4a5d-a35e-84ae64fecac9\"", request_id: "9135bb66-12f7-485a-9e74-46e5276b44e7" } file = sled-agent/src/rack_setup/service.rs:1078
18542024-12-21T01:16:35.335ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "11a61ef2-b359-41be-b83c-9770f0c60add", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:34 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"b1eebc13-191d-4a5d-a35e-84ae64fecac9\"", request_id: "11a61ef2-b359-41be-b83c-9770f0c60add" } file = sled-agent/src/rack_setup/service.rs:1078
18552024-12-21T01:16:36.364ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:36927
18562024-12-21T01:16:36.364ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 28 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:36927 req_id = 5bb71316-12f5-405b-b26f-5ed528aaf01b response_code = 200 uri = /baseboard
18572024-12-21T01:16:38.735ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "8553a3ff-419d-44fc-8b54-a821f8757cd8", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:38 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"b1eebc13-191d-4a5d-a35e-84ae64fecac9\"", request_id: "8553a3ff-419d-44fc-8b54-a821f8757cd8" } file = sled-agent/src/rack_setup/service.rs:1078
18582024-12-21T01:16:40.810ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "1c267ae2-72dc-4c9d-a671-aa829b2ec814", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:40 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"b1eebc13-191d-4a5d-a35e-84ae64fecac9\"", request_id: "1c267ae2-72dc-4c9d-a671-aa829b2ec814" } file = sled-agent/src/rack_setup/service.rs:1078
18592024-12-21T01:16:44.516ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "e61d5b5f-872a-416c-9b97-4ef9fba810ea", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:44 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"b1eebc13-191d-4a5d-a35e-84ae64fecac9\"", request_id: "e61d5b5f-872a-416c-9b97-4ef9fba810ea" } file = sled-agent/src/rack_setup/service.rs:1078
18602024-12-21T01:16:49.857ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "2a97d07f-0d16-420f-896e-021592c93615", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:49 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"b1eebc13-191d-4a5d-a35e-84ae64fecac9\"", request_id: "2a97d07f-0d16-420f-896e-021592c93615" } file = sled-agent/src/rack_setup/service.rs:1078
18612024-12-21T01:16:53.288ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "eb3ed6a7-c371-42ef-a33c-d5a0d9020175", "content-length": "173", "date": "Sat, 21 Dec 2024 01:16:53 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"b1eebc13-191d-4a5d-a35e-84ae64fecac9\"", request_id: "eb3ed6a7-c371-42ef-a33c-d5a0d9020175" } file = sled-agent/src/rack_setup/service.rs:1078
18622024-12-21T01:16:54.042ZINFOSledAgent (NexusNotifierTask): Successfully put SledAgentInfo to nexus file = sled-agent/src/nexus.rs:500 new_generation = 1 old_generation = 1 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
18632024-12-21T01:16:54.134ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47723
18642024-12-21T01:16:54.135ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
18652024-12-21T01:16:54.136ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
18662024-12-21T01:16:54.136ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-zones.json file = common/src/ledger.rs:177
18672024-12-21T01:16:54.137ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-zones.json file = common/src/ledger.rs:177
18682024-12-21T01:16:54.151ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18692024-12-21T01:16:54.151ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff
18702024-12-21T01:16:54.175ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18712024-12-21T01:16:54.175ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_41cc83ed-adbe-492f-ae31-7074b31692b0
18722024-12-21T01:16:54.197ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18732024-12-21T01:16:54.197ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_7e02d0c4-4e5d-440d-94c1-593a67af3118
18742024-12-21T01:16:54.216ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18752024-12-21T01:16:54.216ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50
18762024-12-21T01:16:54.237ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18772024-12-21T01:16:54.237ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_10325d85-ce1b-49aa-8039-7e9482ec8803
18782024-12-21T01:16:54.257ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18792024-12-21T01:16:54.257ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739
18802024-12-21T01:16:54.278ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18812024-12-21T01:16:54.278ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_192cce10-69a8-4ed6-a459-9ca86a7206ec
18822024-12-21T01:16:54.300ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18832024-12-21T01:16:54.300ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_34789575-52b6-411b-9614-90f11629cb04
18842024-12-21T01:16:54.320ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
18852024-12-21T01:16:54.320ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_e80b2509-2264-41c1-be0f-a3173a243a47
18862024-12-21T01:16:54.331ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 197325 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47723 req_id = dfc4835f-6c7c-4272-b6bf-c02e2bc1a8dc response_code = 200 uri = /inventory
18872024-12-21T01:16:57.807ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45421
18882024-12-21T01:16:57.812ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 44a38c79-185d-44b1-a294-e3a088dd4f4f, kind: Service { id: c4364b0f-0b76-413b-9126-391216a47153 }, name: Name("external-dns-c4364b0f-0b76-413b-9126-391216a47153"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cd175b09-5785-415d-8d32-a617204393a9, kind: Service { id: 24428c69-128b-4995-bc7a-addc66912490 }, name: Name("external-dns-24428c69-128b-4995-bc7a-addc66912490"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 190])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 05db974b-0405-4131-ae02-a37152ea980b, kind: Service { id: fe9c2ca3-0266-4081-952b-4cb3f2bf1371 }, name: Name("nexus-fe9c2ca3-0266-4081-952b-4cb3f2bf1371"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 44])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5b5216f1-9871-4dfb-aced-f4b109e9a1c2, kind: Service { id: ee35abf0-6140-47bd-8a0f-da60e1b0cdad }, name: Name("nexus-ee35abf0-6140-47bd-8a0f-da60e1b0cdad"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 654f7fda-6e14-4220-bc54-c5c6d8426d7e, kind: Service { id: bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1 }, name: Name("nexus-bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 234, 194])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
18892024-12-21T01:16:57.812ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18902024-12-21T01:16:57.812ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18912024-12-21T01:16:57.812ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18922024-12-21T01:16:57.812ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18932024-12-21T01:16:57.812ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18942024-12-21T01:16:57.812ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
18952024-12-21T01:16:57.812ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4951 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:45421 req_id = ad711af8-2515-457f-ad0b-7079ea236e82 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18962024-12-21T01:16:57.813ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:1088
18972024-12-21T01:16:57.817ZINFOSledAgent (RSS): Writing ledger to /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
18982024-12-21T01:16:57.818ZINFOSledAgent (RSS): Writing ledger to /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
18992024-12-21T01:16:57.945ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43360
19002024-12-21T01:16:57.946ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 44a38c79-185d-44b1-a294-e3a088dd4f4f, kind: Service { id: c4364b0f-0b76-413b-9126-391216a47153 }, name: Name("external-dns-c4364b0f-0b76-413b-9126-391216a47153"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cd175b09-5785-415d-8d32-a617204393a9, kind: Service { id: 24428c69-128b-4995-bc7a-addc66912490 }, name: Name("external-dns-24428c69-128b-4995-bc7a-addc66912490"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 190])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 05db974b-0405-4131-ae02-a37152ea980b, kind: Service { id: fe9c2ca3-0266-4081-952b-4cb3f2bf1371 }, name: Name("nexus-fe9c2ca3-0266-4081-952b-4cb3f2bf1371"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 44])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5b5216f1-9871-4dfb-aced-f4b109e9a1c2, kind: Service { id: ee35abf0-6140-47bd-8a0f-da60e1b0cdad }, name: Name("nexus-ee35abf0-6140-47bd-8a0f-da60e1b0cdad"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 654f7fda-6e14-4220-bc54-c5c6d8426d7e, kind: Service { id: bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1 }, name: Name("nexus-bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 234, 194])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
19012024-12-21T01:16:57.946ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19022024-12-21T01:16:57.946ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19032024-12-21T01:16:57.946ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19042024-12-21T01:16:57.946ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19052024-12-21T01:16:57.946ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19062024-12-21T01:16:57.946ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
19072024-12-21T01:16:57.946ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 226 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:43360 req_id = 35e60702-b0fc-469f-9614-5beba26578f6 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19082024-12-21T01:16:58.058ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44071
19092024-12-21T01:16:58.058ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
19102024-12-21T01:16:58.058ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
19112024-12-21T01:16:58.058ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-zones.json file = common/src/ledger.rs:177
19122024-12-21T01:16:58.058ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-zones.json file = common/src/ledger.rs:177
19132024-12-21T01:16:58.070ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
19142024-12-21T01:16:58.070ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff
19152024-12-21T01:16:58.087ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53020
19162024-12-21T01:16:58.088ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 44a38c79-185d-44b1-a294-e3a088dd4f4f, kind: Service { id: c4364b0f-0b76-413b-9126-391216a47153 }, name: Name("external-dns-c4364b0f-0b76-413b-9126-391216a47153"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cd175b09-5785-415d-8d32-a617204393a9, kind: Service { id: 24428c69-128b-4995-bc7a-addc66912490 }, name: Name("external-dns-24428c69-128b-4995-bc7a-addc66912490"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 190])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 05db974b-0405-4131-ae02-a37152ea980b, kind: Service { id: fe9c2ca3-0266-4081-952b-4cb3f2bf1371 }, name: Name("nexus-fe9c2ca3-0266-4081-952b-4cb3f2bf1371"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 44])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5b5216f1-9871-4dfb-aced-f4b109e9a1c2, kind: Service { id: ee35abf0-6140-47bd-8a0f-da60e1b0cdad }, name: Name("nexus-ee35abf0-6140-47bd-8a0f-da60e1b0cdad"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 654f7fda-6e14-4220-bc54-c5c6d8426d7e, kind: Service { id: bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1 }, name: Name("nexus-bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 234, 194])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
19172024-12-21T01:16:58.088ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19182024-12-21T01:16:58.088ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19192024-12-21T01:16:58.088ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19202024-12-21T01:16:58.088ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19212024-12-21T01:16:58.088ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19222024-12-21T01:16:58.088ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
19232024-12-21T01:16:58.088ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 586 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53020 req_id = eb18fe2f-fbbc-40cc-893d-1edafec5f222 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19242024-12-21T01:16:58.095ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
19252024-12-21T01:16:58.095ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_41cc83ed-adbe-492f-ae31-7074b31692b0
19262024-12-21T01:16:58.118ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
19272024-12-21T01:16:58.119ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_7e02d0c4-4e5d-440d-94c1-593a67af3118
19282024-12-21T01:16:58.148ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
19292024-12-21T01:16:58.148ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50
19302024-12-21T01:16:58.171ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
19312024-12-21T01:16:58.171ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_10325d85-ce1b-49aa-8039-7e9482ec8803
19322024-12-21T01:16:58.194ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
19332024-12-21T01:16:58.194ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739
19342024-12-21T01:16:58.215ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
19352024-12-21T01:16:58.215ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_192cce10-69a8-4ed6-a459-9ca86a7206ec
19362024-12-21T01:16:58.236ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
19372024-12-21T01:16:58.236ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_34789575-52b6-411b-9614-90f11629cb04
19382024-12-21T01:16:58.259ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
19392024-12-21T01:16:58.259ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_e80b2509-2264-41c1-be0f-a3173a243a47
19402024-12-21T01:16:58.271ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 213136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44071 req_id = c9119c2c-c810-4775-9f7d-dad33f3f044f response_code = 200 uri = /inventory
19412024-12-21T01:16:58.293ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49990
19422024-12-21T01:16:58.293ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 44a38c79-185d-44b1-a294-e3a088dd4f4f, kind: Service { id: c4364b0f-0b76-413b-9126-391216a47153 }, name: Name("external-dns-c4364b0f-0b76-413b-9126-391216a47153"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cd175b09-5785-415d-8d32-a617204393a9, kind: Service { id: 24428c69-128b-4995-bc7a-addc66912490 }, name: Name("external-dns-24428c69-128b-4995-bc7a-addc66912490"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 190])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 05db974b-0405-4131-ae02-a37152ea980b, kind: Service { id: fe9c2ca3-0266-4081-952b-4cb3f2bf1371 }, name: Name("nexus-fe9c2ca3-0266-4081-952b-4cb3f2bf1371"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 44])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5b5216f1-9871-4dfb-aced-f4b109e9a1c2, kind: Service { id: ee35abf0-6140-47bd-8a0f-da60e1b0cdad }, name: Name("nexus-ee35abf0-6140-47bd-8a0f-da60e1b0cdad"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 654f7fda-6e14-4220-bc54-c5c6d8426d7e, kind: Service { id: bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1 }, name: Name("nexus-bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 234, 194])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
19432024-12-21T01:16:58.293ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19442024-12-21T01:16:58.293ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19452024-12-21T01:16:58.293ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19462024-12-21T01:16:58.293ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
19472024-12-21T01:16:58.293ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
19482024-12-21T01:16:58.293ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
19492024-12-21T01:16:58.293ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 228 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:49990 req_id = 6c121549-2b5c-4242-b87d-44d76d191bc5 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
19502024-12-21T01:17:04.600ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43152
19512024-12-21T01:17:04.601ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 532 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43152 req_id = 3573c72f-81cc-42a0-800d-faa07534fe01 response_code = 200 uri = /vpc-routes
19522024-12-21T01:17:04.634ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
19532024-12-21T01:17:04.634ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1724 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43152 req_id = 418d9775-754a-4d35-87e2-c45896da9a8a response_code = 204 uri = /eip-gateways
19542024-12-21T01:17:04.727ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3222 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43152 req_id = 379e783a-79b7-4341-95b8-0a2cb19d5ac8 response_code = 204 uri = /vpc-routes
19552024-12-21T01:17:06.396ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:59754
19562024-12-21T01:17:06.396ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 29 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:59754 req_id = cd6958da-31ac-473f-93ad-03fcb186fde5 response_code = 200 uri = /baseboard
19572024-12-21T01:17:13.384ZINFOSledAgent (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:42313 remote_addr = [fd00:1122:3344:101::d]:61296 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
19582024-12-21T01:17:13.411ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 26363 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = 2e71caa7-e79f-4136-894a-622598c996e5 response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
19592024-12-21T01:17:21.154ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52650
19602024-12-21T01:17:21.154ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52650 req_id = b80b1c23-507c-491c-8bd3-0ee1fb0c1c7d response_code = 200 uri = /vpc-routes
19612024-12-21T01:17:21.216ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
19622024-12-21T01:17:21.216ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:52650 req_id = c508915a-509b-4653-929d-70a2b882040f response_code = 204 uri = /eip-gateways
19632024-12-21T01:17:21.237ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 231 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:52650 req_id = 30f5b473-024d-4325-b347-ed8870e42d6c response_code = 204 uri = /vpc-routes
19642024-12-21T01:17:21.508ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43035
19652024-12-21T01:17:21.508ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 304 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43035 req_id = abb35d75-caec-457e-b277-e652cb95a278 response_code = 200 uri = /v2p
19662024-12-21T01:17:21.703ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60396
19672024-12-21T01:17:21.704ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
19682024-12-21T01:17:21.778ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
19692024-12-21T01:17:21.836ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 132687 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:60396 req_id = f67829ad-65ad-44a4-8c6a-ba64a3a466b0 response_code = 204 uri = /switch-ports
19702024-12-21T01:17:21.839ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2648 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60396 req_id = de8aaedf-f9b3-4267-8214-5c1c61554b1d response_code = 200 uri = /network-bootstore-config
19712024-12-21T01:17:21.881ZINFOSledAgent (bootstore): Attempting to update network config with generation: 2, current_generation: 1 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
19722024-12-21T01:17:21.882ZINFOSledAgent (bootstore): Writing ledger to /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
19732024-12-21T01:17:21.883ZINFOSledAgent (bootstore): Writing ledger to /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
19742024-12-21T01:17:21.884ZINFOSledAgent (bootstore): Broadcasting network config with generation 2 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
19752024-12-21T01:17:21.884ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3824 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:60396 req_id = 1262eddc-0fef-46f7-9546-4a2921e5e16e response_code = 204 uri = /network-bootstore-config
19762024-12-21T01:17:22.476ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51676
19772024-12-21T01:17:22.477ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51676 req_id = 5f2471b9-d4ec-491a-a41a-8f1cbb895694 response_code = 200 uri = /v2p
19782024-12-21T01:17:22.510ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63056
19792024-12-21T01:17:22.511ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63056 req_id = 7ca99850-5d4e-444a-ad00-7f89d02404ce response_code = 200 uri = /vpc-routes
19802024-12-21T01:17:22.538ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
19812024-12-21T01:17:22.538ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:63056 req_id = 4c819f71-2ad2-4629-a02c-b92834069deb response_code = 204 uri = /eip-gateways
19822024-12-21T01:17:22.560ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 278 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:63056 req_id = 2dcdac7a-660b-483a-af36-77713c9b4b9d response_code = 204 uri = /vpc-routes
19832024-12-21T01:17:22.615ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43161
19842024-12-21T01:17:22.615ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43161 req_id = 47062097-b6a5-47a1-9cfc-2b6d74ebd867 response_code = 200 uri = /vpc-routes
19852024-12-21T01:17:22.682ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
19862024-12-21T01:17:22.682ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43161 req_id = add39291-021b-4ca0-9be5-bd43b8300aad response_code = 204 uri = /eip-gateways
19872024-12-21T01:17:22.692ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 175 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43161 req_id = 52428d36-5a99-428e-9fc1-fa81839b27ba response_code = 204 uri = /vpc-routes
19882024-12-21T01:17:22.741ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38927
19892024-12-21T01:17:22.742ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
19902024-12-21T01:17:22.784ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
19912024-12-21T01:17:22.844ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102293 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:38927 req_id = 7e5afb32-d633-4245-a369-cf5a2cde3c72 response_code = 204 uri = /switch-ports
19922024-12-21T01:17:22.844ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39238
19932024-12-21T01:17:22.844ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52149
19942024-12-21T01:17:22.844ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
19952024-12-21T01:17:22.848ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3901 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39238 req_id = 46f037b2-4f14-402e-8c24-f750c9f36c31 response_code = 200 uri = /v2p
19962024-12-21T01:17:22.848ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4094 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38927 req_id = 3f606563-f358-4331-8238-29c972a9f229 response_code = 200 uri = /network-bootstore-config
19972024-12-21T01:17:22.882ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
19982024-12-21T01:17:22.941ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96725 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:52149 req_id = 44ef2b52-6988-443c-897c-95d365245d47 response_code = 204 uri = /switch-ports
19992024-12-21T01:17:22.941ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52149 req_id = ac662c6d-43fd-4859-ac2a-af7748ae1426 response_code = 200 uri = /network-bootstore-config
20002024-12-21T01:17:36.430ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:43357
20012024-12-21T01:17:36.430ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 110 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:43357 req_id = e57b89d5-ee67-416d-b228-a442c568cf28 response_code = 200 uri = /baseboard
20022024-12-21T01:17:43.392ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8371 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = 6fe7b828-d77c-439a-9dbe-47c6ac4b8182 response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
20032024-12-21T01:17:51.184ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51672
20042024-12-21T01:17:51.184ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 50 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51672 req_id = 27196826-11fa-433f-a1a3-33aa05ef04f7 response_code = 200 uri = /vpc-routes
20052024-12-21T01:17:51.243ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
20062024-12-21T01:17:51.243ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 72 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51672 req_id = 48a529aa-bfa2-4fd6-a757-87ec2091c51d response_code = 204 uri = /eip-gateways
20072024-12-21T01:17:51.251ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 157 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51672 req_id = 0b1fa01d-56e0-4d22-aa1e-fe9e273cb2da response_code = 204 uri = /vpc-routes
20082024-12-21T01:17:51.257ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36356
20092024-12-21T01:17:51.257ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
20102024-12-21T01:17:51.299ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
20112024-12-21T01:17:51.363ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106192 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:36356 req_id = 0be65036-6d48-4260-9c46-3ff586de7a32 response_code = 204 uri = /switch-ports
20122024-12-21T01:17:51.363ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58152
20132024-12-21T01:17:51.363ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 240 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58152 req_id = 91e9d17a-4521-482d-ad09-cbf876c7ad00 response_code = 200 uri = /v2p
20142024-12-21T01:17:51.364ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 246 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36356 req_id = f1172f3e-2cf9-4d75-b7ce-0a0c34e8bb18 response_code = 200 uri = /network-bootstore-config
20152024-12-21T01:17:52.288ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52231
20162024-12-21T01:17:52.288ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52231 req_id = aff62f9a-607e-4470-bcad-bc74da72ab4a response_code = 200 uri = /vpc-routes
20172024-12-21T01:17:52.347ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
20182024-12-21T01:17:52.347ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52231 req_id = d4f6eaf3-af9d-4dd3-ad00-f344575dc34a response_code = 204 uri = /eip-gateways
20192024-12-21T01:17:52.355ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 293 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52231 req_id = 91bf9c76-eb9a-4376-aabb-4f9e460c0e8f response_code = 204 uri = /vpc-routes
20202024-12-21T01:17:52.394ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58095
20212024-12-21T01:17:52.394ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58095 req_id = 5d6c7531-3702-4cd8-b77c-a7e248fda928 response_code = 200 uri = /vpc-routes
20222024-12-21T01:17:52.419ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46206
20232024-12-21T01:17:52.419ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46206 req_id = 9f42a909-45cd-4599-9053-2f55a5b61353 response_code = 200 uri = /v2p
20242024-12-21T01:17:52.440ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
20252024-12-21T01:17:52.440ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58095 req_id = 948f18c8-9bc3-41ae-a424-3d5b6e5cfcac response_code = 204 uri = /eip-gateways
20262024-12-21T01:17:52.446ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57956
20272024-12-21T01:17:52.446ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
20282024-12-21T01:17:52.481ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
20292024-12-21T01:17:52.537ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91023 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:57956 req_id = fd0d4400-3f4f-4810-acb6-885be1cb171f response_code = 204 uri = /switch-ports
20302024-12-21T01:17:52.537ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62078
20312024-12-21T01:17:52.537ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 268 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57956 req_id = fbe0f63a-53a2-44f1-b745-1d710cafaefb response_code = 200 uri = /network-bootstore-config
20322024-12-21T01:17:52.537ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 256 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62078 req_id = c4e7dcfb-969d-4da4-8493-94063e712528 response_code = 200 uri = /v2p
20332024-12-21T01:17:52.537ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 401 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58095 req_id = 1150e938-3dc8-4de5-b97a-692851ba0f48 response_code = 204 uri = /vpc-routes
20342024-12-21T01:17:52.546ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64100
20352024-12-21T01:17:52.546ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
20362024-12-21T01:17:52.583ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
20372024-12-21T01:17:52.642ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95938 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:64100 req_id = 1d8d6bcd-b508-494f-958e-e05329e1735e response_code = 204 uri = /switch-ports
20382024-12-21T01:17:52.642ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64100 req_id = 201b21a2-e8cc-416b-990a-44370e95ff57 response_code = 200 uri = /network-bootstore-config
20392024-12-21T01:18:06.463ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:61920
20402024-12-21T01:18:06.463ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 29 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:61920 req_id = 3f195c1b-1fa7-4f2d-a730-23805bbc5be0 response_code = 200 uri = /baseboard
20412024-12-21T01:18:13.392ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8250 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = 4b0c3c1a-08cb-4bf0-a346-e6c27e4b3633 response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
20422024-12-21T01:18:21.488ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36752
20432024-12-21T01:18:21.488ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36752 req_id = df2a52a1-4bce-452a-9416-49b55e0472ce response_code = 200 uri = /v2p
20442024-12-21T01:18:21.539ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50261
20452024-12-21T01:18:21.539ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 33 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50261 req_id = b190c701-b0e2-4eaf-a384-9278eef46451 response_code = 200 uri = /vpc-routes
20462024-12-21T01:18:21.666ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
20472024-12-21T01:18:21.666ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50261 req_id = 9e341df7-5049-4649-affa-8f98c1056ec7 response_code = 204 uri = /eip-gateways
20482024-12-21T01:18:21.773ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 297 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50261 req_id = 0c4a8101-ef04-4c6c-93f2-3ceda14ca986 response_code = 204 uri = /vpc-routes
20492024-12-21T01:18:21.795ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50148
20502024-12-21T01:18:21.795ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
20512024-12-21T01:18:21.831ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
20522024-12-21T01:18:21.881ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86060 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:50148 req_id = 629f0eb2-976a-4610-a275-dea7afd19467 response_code = 204 uri = /switch-ports
20532024-12-21T01:18:21.882ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50148 req_id = 52be5921-b187-4b3a-9924-5f437e63282d response_code = 200 uri = /network-bootstore-config
20542024-12-21T01:18:22.477ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60671
20552024-12-21T01:18:22.478ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 194 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60671 req_id = 62b08401-deb0-41b2-8c77-44e6e1ebfcc3 response_code = 200 uri = /v2p
20562024-12-21T01:18:22.485ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37831
20572024-12-21T01:18:22.485ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37831 req_id = b65a387d-06ed-44bf-85c3-4661917a95a3 response_code = 200 uri = /vpc-routes
20582024-12-21T01:18:22.608ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
20592024-12-21T01:18:22.608ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37831 req_id = 9480492e-de07-4a92-804a-5846cbe3323e response_code = 204 uri = /eip-gateways
20602024-12-21T01:18:22.673ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50184
20612024-12-21T01:18:22.673ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50184 req_id = d7231e43-4927-45c6-8163-e8abb1ab610e response_code = 200 uri = /v2p
20622024-12-21T01:18:22.688ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62728
20632024-12-21T01:18:22.688ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62728 req_id = 5053e5a1-b9b8-4c15-9dd4-da4f80dc3ebd response_code = 200 uri = /vpc-routes
20642024-12-21T01:18:22.717ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 198 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37831 req_id = f15d3760-02db-425b-9b49-de7c33d211d9 response_code = 204 uri = /vpc-routes
20652024-12-21T01:18:22.757ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47780
20662024-12-21T01:18:22.758ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
20672024-12-21T01:18:22.800ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
20682024-12-21T01:18:22.859ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101910 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:47780 req_id = d44482c7-2194-4186-b45a-ede8b2f8a125 response_code = 204 uri = /switch-ports
20692024-12-21T01:18:22.860ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63048
20702024-12-21T01:18:22.860ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
20712024-12-21T01:18:22.860ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 139 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:62728 req_id = 044a4c3e-8434-4878-ad50-7d759b47917e response_code = 204 uri = /eip-gateways
20722024-12-21T01:18:22.860ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 244 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47780 req_id = f8f74ef7-2757-44e7-ba33-014d34e68279 response_code = 200 uri = /network-bootstore-config
20732024-12-21T01:18:22.860ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
20742024-12-21T01:18:22.898ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
20752024-12-21T01:18:22.954ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94407 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:63048 req_id = 154e0a66-8b62-4efc-8de7-fa09e2e5906a response_code = 204 uri = /switch-ports
20762024-12-21T01:18:22.955ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 160 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63048 req_id = e83c9eee-9d74-4ebd-beda-bd355a794d7d response_code = 200 uri = /network-bootstore-config
20772024-12-21T01:18:22.980ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 230 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:62728 req_id = 21549de4-2226-4971-bb4e-a69c7f3b87b3 response_code = 204 uri = /vpc-routes
20782024-12-21T01:18:36.504ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:63116
20792024-12-21T01:18:36.504ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 72 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:63116 req_id = 84ec5764-11c8-4593-8530-d6d7dc80e00b response_code = 200 uri = /baseboard
20802024-12-21T01:18:43.391ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7135 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = 618d4abf-d1ea-441a-bcb5-3ad9ef8ae85f response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
20812024-12-21T01:18:51.231ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51148
20822024-12-21T01:18:51.231ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51148 req_id = 9de38f69-e021-4c7a-a997-b955d9728dd4 response_code = 200 uri = /vpc-routes
20832024-12-21T01:18:51.296ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57409
20842024-12-21T01:18:51.296ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57409 req_id = acee564e-3dd0-4b7e-9f2c-f832093c7131 response_code = 200 uri = /v2p
20852024-12-21T01:18:51.301ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
20862024-12-21T01:18:51.301ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51148 req_id = 619bec9a-f3f2-4ee8-ab92-9eb1e6d659d4 response_code = 204 uri = /eip-gateways
20872024-12-21T01:18:51.357ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47828
20882024-12-21T01:18:51.358ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
20892024-12-21T01:18:51.399ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
20902024-12-21T01:18:51.461ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103029 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:47828 req_id = c39fe9f9-4761-410b-ad19-5504a8bf5f67 response_code = 204 uri = /switch-ports
20912024-12-21T01:18:51.461ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 277 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47828 req_id = a9d45e29-3b7f-4bf6-bae0-5ea66607e1dc response_code = 200 uri = /network-bootstore-config
20922024-12-21T01:18:51.461ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 471 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51148 req_id = 492e42b4-7027-4cb8-850a-9ac091df57b5 response_code = 204 uri = /vpc-routes
20932024-12-21T01:18:52.346ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35253
20942024-12-21T01:18:52.346ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35253 req_id = c80edb71-d88a-4cb0-a7e3-f4521b051e22 response_code = 200 uri = /vpc-routes
20952024-12-21T01:18:52.388ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
20962024-12-21T01:18:52.388ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35253 req_id = 7b451e5a-87a5-4b00-baa9-0950730939e7 response_code = 204 uri = /eip-gateways
20972024-12-21T01:18:52.399ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 176 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:35253 req_id = be9e2c29-8b0b-4da3-b8c2-dd32a8e84569 response_code = 204 uri = /vpc-routes
20982024-12-21T01:18:52.429ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56697
20992024-12-21T01:18:52.429ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 34 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56697 req_id = 80e5bfa9-7d88-43d7-a3b9-7e56c9770c7c response_code = 200 uri = /vpc-routes
21002024-12-21T01:18:52.449ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36182
21012024-12-21T01:18:52.450ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 139 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36182 req_id = b5e383c4-9869-448b-9b80-2780dac6d8e5 response_code = 200 uri = /v2p
21022024-12-21T01:18:52.450ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48001
21032024-12-21T01:18:52.450ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 273 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48001 req_id = f3d97a39-2a90-4706-ab97-751a5aa792fd response_code = 200 uri = /v2p
21042024-12-21T01:18:52.521ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42259
21052024-12-21T01:18:52.521ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
21062024-12-21T01:18:52.565ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
21072024-12-21T01:18:52.630ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 109898 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:42259 req_id = a2e08052-59c1-4aa5-bab5-b475d99ee2ab response_code = 204 uri = /switch-ports
21082024-12-21T01:18:52.631ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38773
21092024-12-21T01:18:52.631ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
21102024-12-21T01:18:52.631ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 141 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:56697 req_id = 231dcfbb-9381-4652-a83c-675762070faf response_code = 204 uri = /eip-gateways
21112024-12-21T01:18:52.631ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 228 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42259 req_id = 06fee49b-223b-42dd-a0aa-0c306e12a7ab response_code = 200 uri = /network-bootstore-config
21122024-12-21T01:18:52.631ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
21132024-12-21T01:18:52.641ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 272 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:56697 req_id = e479257f-c163-43ad-a649-e60bc0f2f044 response_code = 204 uri = /vpc-routes
21142024-12-21T01:18:52.670ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
21152024-12-21T01:18:52.734ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102576 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:38773 req_id = c8795009-9395-4d3a-8515-7c179c31be7d response_code = 204 uri = /switch-ports
21162024-12-21T01:18:52.734ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 152 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38773 req_id = 218c45fe-64b7-412a-82e3-5a757dccd02e response_code = 200 uri = /network-bootstore-config
21172024-12-21T01:19:06.538ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:57225
21182024-12-21T01:19:06.538ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 30 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:57225 req_id = a0f862e3-b942-4f03-9c75-f990e16bd9bc response_code = 200 uri = /baseboard
21192024-12-21T01:19:13.391ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7529 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = b4b0c7ca-7fd9-44d9-959e-2c4d7ba55b04 response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
21202024-12-21T01:19:21.468ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57647
21212024-12-21T01:19:21.468ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 138 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57647 req_id = ccf44a41-affd-40e8-bb79-f2e974dcd268 response_code = 200 uri = /v2p
21222024-12-21T01:19:21.475ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52703
21232024-12-21T01:19:21.475ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 137 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52703 req_id = c700763c-f70e-4589-a052-e558d9e75446 response_code = 200 uri = /vpc-routes
21242024-12-21T01:19:21.526ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
21252024-12-21T01:19:21.526ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:52703 req_id = 0da23f8e-5cd4-48ae-bc0c-bc1500c2eecd response_code = 204 uri = /eip-gateways
21262024-12-21T01:19:21.534ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 238 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:52703 req_id = b84cf1c1-036c-4ace-8bba-96e20c8aa024 response_code = 204 uri = /vpc-routes
21272024-12-21T01:19:21.795ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58734
21282024-12-21T01:19:21.795ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
21292024-12-21T01:19:21.833ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
21302024-12-21T01:19:21.886ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90866 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:58734 req_id = 8939c3dc-8c5e-4aba-80c9-1c3b6f06e77b response_code = 204 uri = /switch-ports
21312024-12-21T01:19:21.886ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58734 req_id = 399928ab-a9a1-4b82-bbcd-d83918a73e68 response_code = 200 uri = /network-bootstore-config
21322024-12-21T01:19:22.428ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
21332024-12-21T01:19:22.462ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33669
21342024-12-21T01:19:22.462ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33669 req_id = 41c00c4e-eee8-46cd-b768-db36b18260b7 response_code = 200 uri = /vpc-routes
21352024-12-21T01:19:22.515ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
21362024-12-21T01:19:22.515ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:33669 req_id = 539f7ae9-8b6f-464d-9117-5137bea118ed response_code = 204 uri = /eip-gateways
21372024-12-21T01:19:22.529ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 192 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:33669 req_id = 93d6c5e8-d521-4c53-8d24-c58969c397d2 response_code = 204 uri = /vpc-routes
21382024-12-21T01:19:22.611ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56999
21392024-12-21T01:19:22.612ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56999 req_id = bbe58843-b724-4db2-9d0d-1bcd5065697a response_code = 200 uri = /v2p
21402024-12-21T01:19:22.618ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47563
21412024-12-21T01:19:22.618ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 34 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47563 req_id = 8ceab606-d8c9-45ec-af0d-83ce4235650e response_code = 200 uri = /vpc-routes
21422024-12-21T01:19:22.664ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
21432024-12-21T01:19:22.664ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47563 req_id = d3c97cfe-3c48-4f1a-9ab7-9db601430c5f response_code = 204 uri = /eip-gateways
21442024-12-21T01:19:22.672ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 230 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47563 req_id = 8286494e-0f53-434c-950d-5f6c59ea60fc response_code = 204 uri = /vpc-routes
21452024-12-21T01:19:22.730ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38984
21462024-12-21T01:19:22.730ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
21472024-12-21T01:19:22.764ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
21482024-12-21T01:19:22.820ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90179 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:38984 req_id = 689c6a48-ee99-46d6-8a04-e3d9537e145e response_code = 204 uri = /switch-ports
21492024-12-21T01:19:22.820ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44278
21502024-12-21T01:19:22.821ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 188 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:44278 req_id = 57c6ae7c-f043-4009-a94b-ed8939be941e response_code = 200 uri = /v2p
21512024-12-21T01:19:22.821ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 224 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38984 req_id = 3f4aa58a-a819-4179-a74a-4103976bfb8a response_code = 200 uri = /network-bootstore-config
21522024-12-21T01:19:22.833ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63488
21532024-12-21T01:19:22.833ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
21542024-12-21T01:19:22.871ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
21552024-12-21T01:19:22.925ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91653 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:63488 req_id = 94e0ae78-abdf-4299-9a78-32d829647ac6 response_code = 204 uri = /switch-ports
21562024-12-21T01:19:22.925ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63488 req_id = 7284b69b-f711-4bcf-9c79-d32a15b463f9 response_code = 200 uri = /network-bootstore-config
21572024-12-21T01:19:36.572ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:37978
21582024-12-21T01:19:36.572ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 28 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:37978 req_id = 93e63f79-8391-4e85-938b-b98670ab48e9 response_code = 200 uri = /baseboard
21592024-12-21T01:19:43.392ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8489 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = 121dd3d7-a3e9-47c3-92b4-9c9541cc7852 response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
21602024-12-21T01:19:51.255ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56036
21612024-12-21T01:19:51.255ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56036 req_id = efd7a5e4-201e-42fd-8ba1-cd04faa21dac response_code = 200 uri = /vpc-routes
21622024-12-21T01:19:51.309ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
21632024-12-21T01:19:51.309ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56036 req_id = 0369fdf3-2578-44dd-8272-492f74df8d18 response_code = 204 uri = /eip-gateways
21642024-12-21T01:19:51.315ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44860
21652024-12-21T01:19:51.315ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
21662024-12-21T01:19:51.356ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
21672024-12-21T01:19:51.414ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99411 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:44860 req_id = d5edb5c1-4cc6-463f-a3eb-6eb9b2cdfda9 response_code = 204 uri = /switch-ports
21682024-12-21T01:19:51.414ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43578
21692024-12-21T01:19:51.415ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 333 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44860 req_id = 5180fc4f-0470-498b-844b-86e2b51c4975 response_code = 200 uri = /network-bootstore-config
21702024-12-21T01:19:51.415ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 493 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56036 req_id = 2c609a25-a844-4bbf-a2f2-9b7eff339fdd response_code = 204 uri = /vpc-routes
21712024-12-21T01:19:51.415ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 292 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43578 req_id = 480793fe-f3dd-4cca-a10e-a395dc20cd87 response_code = 200 uri = /v2p
21722024-12-21T01:19:52.440ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:65008
21732024-12-21T01:19:52.440ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:65008 req_id = d7ced313-87d9-4fed-a92b-a700edd60680 response_code = 200 uri = /v2p
21742024-12-21T01:19:52.443ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50473
21752024-12-21T01:19:52.443ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 39 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50473 req_id = 1d8a986c-e6dc-4294-b5c7-baada53fc883 response_code = 200 uri = /vpc-routes
21762024-12-21T01:19:52.475ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52841
21772024-12-21T01:19:52.475ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 33 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52841 req_id = 3b853b80-15fc-4e49-934e-ad9c7bb653b2 response_code = 200 uri = /vpc-routes
21782024-12-21T01:19:52.483ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58692
21792024-12-21T01:19:52.483ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 139 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58692 req_id = bd0db65d-5189-4cfb-aa2d-60e32743173e response_code = 200 uri = /v2p
21802024-12-21T01:19:52.488ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
21812024-12-21T01:19:52.488ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:50473 req_id = cc7e0b3a-acb6-4fbf-842d-ef9da0f7f0ee response_code = 204 uri = /eip-gateways
21822024-12-21T01:19:52.490ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
21832024-12-21T01:19:52.490ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 31 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52841 req_id = e557c739-1062-4bd2-b528-7653fe4ae320 response_code = 204 uri = /eip-gateways
21842024-12-21T01:19:52.499ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 266 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52841 req_id = d5b7f36e-6b07-4ea6-9126-414d2931db74 response_code = 204 uri = /vpc-routes
21852024-12-21T01:19:52.602ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 290 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:50473 req_id = 3de8b16e-e5ef-4603-a034-1685f65b7374 response_code = 204 uri = /vpc-routes
21862024-12-21T01:19:52.643ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35133
21872024-12-21T01:19:52.643ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
21882024-12-21T01:19:52.684ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
21892024-12-21T01:19:52.742ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99100 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:35133 req_id = 6d4bb2ae-e41f-4d3b-ac1b-9a09c7dff3c7 response_code = 204 uri = /switch-ports
21902024-12-21T01:19:52.743ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45395
21912024-12-21T01:19:52.743ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
21922024-12-21T01:19:52.764ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 158 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35133 req_id = eaf7f27d-f4f9-4549-b63e-2a282cf2a7fe response_code = 200 uri = /network-bootstore-config
21932024-12-21T01:19:52.778ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
21942024-12-21T01:19:52.840ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97794 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:45395 req_id = be87f636-6fae-4704-8aa5-5d14a5150d49 response_code = 204 uri = /switch-ports
21952024-12-21T01:19:52.841ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 160 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45395 req_id = 2e01b0a4-7680-479c-94a6-e5dd850f3f55 response_code = 200 uri = /network-bootstore-config
21962024-12-21T01:20:06.604ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47370
21972024-12-21T01:20:06.604ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 29 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47370 req_id = 5f1617a0-b065-4949-9c15-2ea776c31ae3 response_code = 200 uri = /baseboard
21982024-12-21T01:20:13.392ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8300 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = facb588e-7f5f-4d44-80fd-78273733ec12 response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
21992024-12-21T01:20:21.469ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64243
22002024-12-21T01:20:21.469ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64243 req_id = 621e255b-de47-4c43-8092-f205536e7dfb response_code = 200 uri = /vpc-routes
22012024-12-21T01:20:21.521ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
22022024-12-21T01:20:21.522ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 80 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:64243 req_id = b47e50aa-8cd7-401d-b384-0710024ee7df response_code = 204 uri = /eip-gateways
22032024-12-21T01:20:21.531ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 179 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:64243 req_id = f01b33c4-f812-4c22-a64c-6621cd5a0d14 response_code = 204 uri = /vpc-routes
22042024-12-21T01:20:21.592ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48498
22052024-12-21T01:20:21.592ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48498 req_id = d735495e-e749-481c-be86-c312748be164 response_code = 200 uri = /v2p
22062024-12-21T01:20:21.610ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34335
22072024-12-21T01:20:21.610ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
22082024-12-21T01:20:21.650ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
22092024-12-21T01:20:21.705ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95328 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:34335 req_id = fa905879-5f50-4307-a8bd-c7ee166a8e6d response_code = 204 uri = /switch-ports
22102024-12-21T01:20:21.705ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34335 req_id = 4a9de54b-29c1-4d96-b954-98699bdee5c9 response_code = 200 uri = /network-bootstore-config
22112024-12-21T01:20:22.391ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62140
22122024-12-21T01:20:22.391ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62140 req_id = cf216539-25cc-4254-8c53-075ce50912de response_code = 200 uri = /vpc-routes
22132024-12-21T01:20:22.449ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
22142024-12-21T01:20:22.449ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:62140 req_id = 30218e74-7c0e-486b-a773-1b7608e9521c response_code = 204 uri = /eip-gateways
22152024-12-21T01:20:22.554ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 204 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:62140 req_id = 67a7c2e3-8465-4456-a756-c09bf82ad41f response_code = 204 uri = /vpc-routes
22162024-12-21T01:20:22.574ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43474
22172024-12-21T01:20:22.574ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 119 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43474 req_id = 9ef80cb6-bd45-417f-889e-00d1fd8856b1 response_code = 200 uri = /v2p
22182024-12-21T01:20:22.579ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59147
22192024-12-21T01:20:22.579ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59147 req_id = 136173a5-b9d9-434b-aab7-e5f6ce098b60 response_code = 200 uri = /vpc-routes
22202024-12-21T01:20:22.585ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43133
22212024-12-21T01:20:22.585ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43133 req_id = 8131cffe-a869-406f-8759-af85c561732a response_code = 200 uri = /v2p
22222024-12-21T01:20:22.731ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
22232024-12-21T01:20:22.731ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59147 req_id = 63eee2c1-de9f-4d9b-9a26-2e9c18bbe8a7 response_code = 204 uri = /eip-gateways
22242024-12-21T01:20:22.742ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 176 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59147 req_id = 9462f819-9949-4140-8ceb-096d6e81c2f3 response_code = 204 uri = /vpc-routes
22252024-12-21T01:20:22.754ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:32946
22262024-12-21T01:20:22.754ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
22272024-12-21T01:20:22.791ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
22282024-12-21T01:20:22.849ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95331 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:32946 req_id = 449c0c38-7915-43d2-a40b-e0e106075201 response_code = 204 uri = /switch-ports
22292024-12-21T01:20:22.850ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49185
22302024-12-21T01:20:22.850ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
22312024-12-21T01:20:22.850ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 261 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:32946 req_id = 851a34a6-8821-4b43-86dd-a683a6648966 response_code = 200 uri = /network-bootstore-config
22322024-12-21T01:20:22.887ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
22332024-12-21T01:20:22.948ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98523 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:49185 req_id = a262cae4-4bb0-4c0f-afee-1f69335a82fd response_code = 204 uri = /switch-ports
22342024-12-21T01:20:22.949ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49185 req_id = 823388d4-2ad4-4925-b1b5-847aa992f483 response_code = 200 uri = /network-bootstore-config
22352024-12-21T01:20:36.637ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47319
22362024-12-21T01:20:36.637ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47319 req_id = dafd9c4e-72a8-489c-9630-1eeeb80b3ca6 response_code = 200 uri = /baseboard
22372024-12-21T01:20:43.392ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8112 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = 7cc30c3e-a289-47a7-8e71-dfe3b18dff09 response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
22382024-12-21T01:20:51.187ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56395
22392024-12-21T01:20:51.187ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56395 req_id = 8c3a2d2f-59e7-45b7-a6cb-de6375b2dcb2 response_code = 200 uri = /v2p
22402024-12-21T01:20:51.231ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58340
22412024-12-21T01:20:51.231ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58340 req_id = 23117a52-a4c4-4ead-a553-b2f956254247 response_code = 200 uri = /vpc-routes
22422024-12-21T01:20:51.254ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
22432024-12-21T01:20:51.254ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58340 req_id = 5b971de8-62d2-48d6-b7f0-6d48d03aa8eb response_code = 204 uri = /eip-gateways
22442024-12-21T01:20:51.261ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 170 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58340 req_id = 67f09731-12b0-407b-afd2-94842124e320 response_code = 204 uri = /vpc-routes
22452024-12-21T01:20:51.327ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34935
22462024-12-21T01:20:51.327ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
22472024-12-21T01:20:51.365ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
22482024-12-21T01:20:51.421ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93084 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:34935 req_id = e6a6d7b8-6e57-480c-a222-601bd2189016 response_code = 204 uri = /switch-ports
22492024-12-21T01:20:51.421ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34935 req_id = 4065fce5-10f5-4367-8253-33610e6a8cb2 response_code = 200 uri = /network-bootstore-config
22502024-12-21T01:20:52.254ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48047
22512024-12-21T01:20:52.254ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48047 req_id = ee024a81-8c0f-4e33-bc51-ed84e0c27b57 response_code = 200 uri = /vpc-routes
22522024-12-21T01:20:52.321ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
22532024-12-21T01:20:52.321ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48047 req_id = ed94f52f-2dd4-428f-a646-d3ad9ac90525 response_code = 204 uri = /eip-gateways
22542024-12-21T01:20:52.331ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 275 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:48047 req_id = a4cdac9d-3c31-43dd-94c0-6bd00c984d32 response_code = 204 uri = /vpc-routes
22552024-12-21T01:20:52.380ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:65496
22562024-12-21T01:20:52.380ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 128 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:65496 req_id = 5a5c85b6-3f86-427a-bacf-76c9294cc58f response_code = 200 uri = /v2p
22572024-12-21T01:20:52.448ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:32953
22582024-12-21T01:20:52.449ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
22592024-12-21T01:20:52.484ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
22602024-12-21T01:20:52.535ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86733 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:32953 req_id = 2e9f1d6a-b47f-4cd0-a08d-0f6ccaa7783d response_code = 204 uri = /switch-ports
22612024-12-21T01:20:52.535ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40757
22622024-12-21T01:20:52.536ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 162 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:32953 req_id = ef6f45da-c9c4-43a5-9f1f-4a6b304c2e56 response_code = 200 uri = /network-bootstore-config
22632024-12-21T01:20:52.536ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 132 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40757 req_id = 68ab5175-55e0-49f8-b7e8-f583a4c5977f response_code = 200 uri = /vpc-routes
22642024-12-21T01:20:52.551ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
22652024-12-21T01:20:52.551ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 147 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:40757 req_id = a210f201-a0a2-44f1-9cb9-71ec1fae53e0 response_code = 204 uri = /eip-gateways
22662024-12-21T01:20:52.567ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:65350
22672024-12-21T01:20:52.567ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:65350 req_id = 687c7fbb-e46a-45dd-a545-fa4d091dae5b response_code = 200 uri = /v2p
22682024-12-21T01:20:52.585ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48943
22692024-12-21T01:20:52.585ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
22702024-12-21T01:20:52.623ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
22712024-12-21T01:20:52.677ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92523 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:48943 req_id = e21a4f85-db48-4df7-b30b-48c352596392 response_code = 204 uri = /switch-ports
22722024-12-21T01:20:52.678ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 258 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:40757 req_id = 585f0fbc-a330-41b0-ab5e-e6bcf17f840d response_code = 204 uri = /vpc-routes
22732024-12-21T01:20:52.678ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48943 req_id = 6fb2d307-1d04-4a5c-8221-1f0fed543f29 response_code = 200 uri = /network-bootstore-config
22742024-12-21T01:21:06.671ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:34683
22752024-12-21T01:21:06.671ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 34 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:34683 req_id = 57da8191-0eb1-4373-8480-c1f998a02c84 response_code = 200 uri = /baseboard
22762024-12-21T01:21:13.392ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8134 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = c8369f1f-09fb-4003-b793-96b56e565a32 response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
22772024-12-21T01:21:21.344ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65164
22782024-12-21T01:21:21.344ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65164 req_id = be3a0711-a140-4972-9e61-a9448fa4da4d response_code = 200 uri = /vpc-routes
22792024-12-21T01:21:21.387ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
22802024-12-21T01:21:21.387ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:65164 req_id = e69f2a60-53f4-4f09-958e-a57996f8892c response_code = 204 uri = /eip-gateways
22812024-12-21T01:21:21.395ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 248 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:65164 req_id = 860dd727-9587-4459-a7ab-45a8ebaecd63 response_code = 204 uri = /vpc-routes
22822024-12-21T01:21:21.535ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55954
22832024-12-21T01:21:21.535ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55954 req_id = 4b1f6b70-bb90-4b0f-863b-ec9f82b483f8 response_code = 200 uri = /v2p
22842024-12-21T01:21:21.717ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:61201
22852024-12-21T01:21:21.717ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
22862024-12-21T01:21:21.759ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
22872024-12-21T01:21:21.815ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98294 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:61201 req_id = ebeb33d2-f1c3-4347-8a7c-3ab1abbc3094 response_code = 204 uri = /switch-ports
22882024-12-21T01:21:21.815ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50783
22892024-12-21T01:21:21.815ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:61201 req_id = cf6c538f-0ee0-4c66-bb40-24634ebf36dd response_code = 200 uri = /network-bootstore-config
22902024-12-21T01:21:21.815ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 44a38c79-185d-44b1-a294-e3a088dd4f4f, kind: Service { id: c4364b0f-0b76-413b-9126-391216a47153 }, name: Name("external-dns-c4364b0f-0b76-413b-9126-391216a47153"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cd175b09-5785-415d-8d32-a617204393a9, kind: Service { id: 24428c69-128b-4995-bc7a-addc66912490 }, name: Name("external-dns-24428c69-128b-4995-bc7a-addc66912490"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 190])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 05db974b-0405-4131-ae02-a37152ea980b, kind: Service { id: fe9c2ca3-0266-4081-952b-4cb3f2bf1371 }, name: Name("nexus-fe9c2ca3-0266-4081-952b-4cb3f2bf1371"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 44])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5b5216f1-9871-4dfb-aced-f4b109e9a1c2, kind: Service { id: ee35abf0-6140-47bd-8a0f-da60e1b0cdad }, name: Name("nexus-ee35abf0-6140-47bd-8a0f-da60e1b0cdad"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 654f7fda-6e14-4220-bc54-c5c6d8426d7e, kind: Service { id: bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1 }, name: Name("nexus-bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 234, 194])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
22912024-12-21T01:21:21.816ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22922024-12-21T01:21:21.816ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
22932024-12-21T01:21:21.816ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22942024-12-21T01:21:21.816ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22952024-12-21T01:21:21.816ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
22962024-12-21T01:21:21.816ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
22972024-12-21T01:21:21.816ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 412 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50783 req_id = 9164e202-a53c-40af-80a2-caa39b50c3e2 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
22982024-12-21T01:21:22.452ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44220
22992024-12-21T01:21:22.452ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44220 req_id = c57a87dc-edd0-46aa-9ddb-fc90fe3b0404 response_code = 200 uri = /vpc-routes
23002024-12-21T01:21:22.507ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
23012024-12-21T01:21:22.507ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44220 req_id = fce22c3d-8643-4931-91b3-e1e73a758be4 response_code = 204 uri = /eip-gateways
23022024-12-21T01:21:22.517ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 217 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44220 req_id = eade0d0f-0838-409b-ba20-df5c2762ed7e response_code = 204 uri = /vpc-routes
23032024-12-21T01:21:22.585ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52465
23042024-12-21T01:21:22.585ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52465 req_id = a5522e66-b9b8-4bb7-93cf-d345373975ee response_code = 200 uri = /vpc-routes
23052024-12-21T01:21:22.645ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
23062024-12-21T01:21:22.645ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 72 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52465 req_id = 1d799b2f-f188-424b-bcef-1e789af397e4 response_code = 204 uri = /eip-gateways
23072024-12-21T01:21:22.657ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 284 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52465 req_id = efabc10a-e3c1-44aa-b7a2-17c885a30f30 response_code = 204 uri = /vpc-routes
23082024-12-21T01:21:22.687ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41271
23092024-12-21T01:21:22.687ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41271 req_id = 34955461-d9d8-4288-ad0f-05721d249d4f response_code = 200 uri = /v2p
23102024-12-21T01:21:22.703ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49818
23112024-12-21T01:21:22.704ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
23122024-12-21T01:21:22.746ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
23132024-12-21T01:21:22.807ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103999 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:49818 req_id = 469c53d0-414b-4684-8005-8adf5a6dd731 response_code = 204 uri = /switch-ports
23142024-12-21T01:21:22.807ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48616
23152024-12-21T01:21:22.807ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37157
23162024-12-21T01:21:22.808ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
23172024-12-21T01:21:22.808ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 245 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49818 req_id = 01d83b93-c19e-4ca7-9af6-11cea25ea130 response_code = 200 uri = /network-bootstore-config
23182024-12-21T01:21:22.811ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3801 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48616 req_id = 3bd55d34-93ae-4568-bdfa-d7e34d777d35 response_code = 200 uri = /v2p
23192024-12-21T01:21:22.846ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
23202024-12-21T01:21:22.852ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55729
23212024-12-21T01:21:22.852ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 44a38c79-185d-44b1-a294-e3a088dd4f4f, kind: Service { id: c4364b0f-0b76-413b-9126-391216a47153 }, name: Name("external-dns-c4364b0f-0b76-413b-9126-391216a47153"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cd175b09-5785-415d-8d32-a617204393a9, kind: Service { id: 24428c69-128b-4995-bc7a-addc66912490 }, name: Name("external-dns-24428c69-128b-4995-bc7a-addc66912490"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 190])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 05db974b-0405-4131-ae02-a37152ea980b, kind: Service { id: fe9c2ca3-0266-4081-952b-4cb3f2bf1371 }, name: Name("nexus-fe9c2ca3-0266-4081-952b-4cb3f2bf1371"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 44])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5b5216f1-9871-4dfb-aced-f4b109e9a1c2, kind: Service { id: ee35abf0-6140-47bd-8a0f-da60e1b0cdad }, name: Name("nexus-ee35abf0-6140-47bd-8a0f-da60e1b0cdad"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 654f7fda-6e14-4220-bc54-c5c6d8426d7e, kind: Service { id: bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1 }, name: Name("nexus-bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 234, 194])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
23222024-12-21T01:21:22.852ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23232024-12-21T01:21:22.852ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23242024-12-21T01:21:22.852ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23252024-12-21T01:21:22.852ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23262024-12-21T01:21:22.852ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23272024-12-21T01:21:22.852ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
23282024-12-21T01:21:22.852ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 405 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:55729 req_id = 7fd628ca-4223-48f4-bb9f-c57ea8d4dcb7 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23292024-12-21T01:21:22.904ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96404 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:37157 req_id = 77fc3987-1870-4985-815b-50277979484f response_code = 204 uri = /switch-ports
23302024-12-21T01:21:22.904ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 139 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37157 req_id = 4a779484-6291-4829-9be4-69e139bcd16a response_code = 200 uri = /network-bootstore-config
23312024-12-21T01:21:22.972ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41075
23322024-12-21T01:21:22.972ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 44a38c79-185d-44b1-a294-e3a088dd4f4f, kind: Service { id: c4364b0f-0b76-413b-9126-391216a47153 }, name: Name("external-dns-c4364b0f-0b76-413b-9126-391216a47153"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cd175b09-5785-415d-8d32-a617204393a9, kind: Service { id: 24428c69-128b-4995-bc7a-addc66912490 }, name: Name("external-dns-24428c69-128b-4995-bc7a-addc66912490"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 190])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 05db974b-0405-4131-ae02-a37152ea980b, kind: Service { id: fe9c2ca3-0266-4081-952b-4cb3f2bf1371 }, name: Name("nexus-fe9c2ca3-0266-4081-952b-4cb3f2bf1371"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 44])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5b5216f1-9871-4dfb-aced-f4b109e9a1c2, kind: Service { id: ee35abf0-6140-47bd-8a0f-da60e1b0cdad }, name: Name("nexus-ee35abf0-6140-47bd-8a0f-da60e1b0cdad"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 654f7fda-6e14-4220-bc54-c5c6d8426d7e, kind: Service { id: bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1 }, name: Name("nexus-bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 234, 194])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
23332024-12-21T01:21:22.972ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23342024-12-21T01:21:22.972ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23352024-12-21T01:21:22.972ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23362024-12-21T01:21:22.972ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23372024-12-21T01:21:22.972ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23382024-12-21T01:21:22.972ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
23392024-12-21T01:21:22.972ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 310 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:41075 req_id = f3cd4c66-2a24-4419-bc3b-1335bd0e98c2 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23402024-12-21T01:21:36.705ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:33125
23412024-12-21T01:21:36.705ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:33125 req_id = 0fbc4eda-1270-497f-9b40-c0b4560428a4 response_code = 200 uri = /baseboard
23422024-12-21T01:21:43.391ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7116 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = 310bba38-63db-49e3-8dd7-f341b238a01c response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
23432024-12-21T01:21:51.231ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43209
23442024-12-21T01:21:51.232ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43209 req_id = 2a12ea87-7713-443f-9f36-e099e32f35c4 response_code = 200 uri = /vpc-routes
23452024-12-21T01:21:51.247ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
23462024-12-21T01:21:51.247ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:43209 req_id = 5e3e796f-c156-4547-8fc5-0dc66a8742ba response_code = 204 uri = /eip-gateways
23472024-12-21T01:21:51.332ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57577
23482024-12-21T01:21:51.332ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57577 req_id = da1a77ec-cc69-4927-b0af-bc2734edec78 response_code = 200 uri = /v2p
23492024-12-21T01:21:51.346ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36753
23502024-12-21T01:21:51.346ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
23512024-12-21T01:21:51.384ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
23522024-12-21T01:21:51.441ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94518 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:36753 req_id = e7678887-3872-42d1-9a16-8a080d79e6c1 response_code = 204 uri = /switch-ports
23532024-12-21T01:21:51.441ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 327 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:43209 req_id = dad3b0be-6afd-47f0-8553-f856cb4df9cc response_code = 204 uri = /vpc-routes
23542024-12-21T01:21:51.441ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36753 req_id = 12d618dc-dc5b-4863-ab2d-1652b3d9fe54 response_code = 200 uri = /network-bootstore-config
23552024-12-21T01:21:52.372ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37098
23562024-12-21T01:21:52.372ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37098 req_id = d069ff2e-2374-4ba0-80dd-8b4dd793dd14 response_code = 200 uri = /vpc-routes
23572024-12-21T01:21:52.415ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
23582024-12-21T01:21:52.415ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37098 req_id = a97a1a5a-f1dd-4e9f-9847-0f297d855294 response_code = 204 uri = /eip-gateways
23592024-12-21T01:21:52.427ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49578
23602024-12-21T01:21:52.427ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49578 req_id = 893ce745-f3ed-4cf8-b895-b909657cf588 response_code = 200 uri = /v2p
23612024-12-21T01:21:52.430ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 211 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37098 req_id = 2f0cbe72-c77c-40cc-9bdd-5b08fa1cdd4a response_code = 204 uri = /vpc-routes
23622024-12-21T01:21:52.461ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40988
23632024-12-21T01:21:52.461ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40988 req_id = 412b1da4-0fda-42e0-bd1a-286a3f9bdf1e response_code = 200 uri = /v2p
23642024-12-21T01:21:52.482ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:57597
23652024-12-21T01:21:52.482ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
23662024-12-21T01:21:52.520ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
23672024-12-21T01:21:52.577ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94726 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:57597 req_id = 396057a7-2959-41a9-9701-43f9fae25d86 response_code = 204 uri = /switch-ports
23682024-12-21T01:21:52.577ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48515
23692024-12-21T01:21:52.577ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48515 req_id = 5f5292d8-e385-4035-8279-7bbbe9896848 response_code = 200 uri = /vpc-routes
23702024-12-21T01:21:52.578ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 165 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:57597 req_id = 82e71b9f-11ef-4ba9-91f8-b9fa788aabdb response_code = 200 uri = /network-bootstore-config
23712024-12-21T01:21:52.593ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
23722024-12-21T01:21:52.593ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48515 req_id = 3053b4ff-012d-4053-8964-e38a4c955a8b response_code = 204 uri = /eip-gateways
23732024-12-21T01:21:52.601ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 293 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48515 req_id = fb09e57d-a429-4d21-9fe5-8c3e4ab5875e response_code = 204 uri = /vpc-routes
23742024-12-21T01:21:52.616ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50841
23752024-12-21T01:21:52.616ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
23762024-12-21T01:21:52.651ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
23772024-12-21T01:21:52.702ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86148 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:50841 req_id = 19c2466f-fe83-4d6f-a38a-58c6afb1d8e5 response_code = 204 uri = /switch-ports
23782024-12-21T01:21:52.702ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 117 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50841 req_id = 6ecea66c-7462-4a7f-9e25-201ca630cf4e response_code = 200 uri = /network-bootstore-config
23792024-12-21T01:22:06.740ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:45659
23802024-12-21T01:22:06.740ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:45659 req_id = 7cee3112-0ff6-44ec-8612-827183fd8727 response_code = 200 uri = /baseboard
23812024-12-21T01:22:13.391ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7889 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = 525e861f-e7a5-4c8f-bc3c-c8dbec9bb9db response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
23822024-12-21T01:22:21.334ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36247
23832024-12-21T01:22:21.334ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 49 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36247 req_id = fab603df-a7c4-4184-a9ca-80b0b7f72763 response_code = 200 uri = /vpc-routes
23842024-12-21T01:22:21.369ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48679
23852024-12-21T01:22:21.369ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48679 req_id = 47b922d9-f0da-4388-83aa-867e1e49e660 response_code = 200 uri = /v2p
23862024-12-21T01:22:21.378ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
23872024-12-21T01:22:21.378ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36247 req_id = 5390b0a4-6ff0-4031-91ab-88429c1487c2 response_code = 204 uri = /eip-gateways
23882024-12-21T01:22:21.387ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 227 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36247 req_id = 1fda62cf-3fe7-47ed-9bb1-5ff7228656db response_code = 204 uri = /vpc-routes
23892024-12-21T01:22:21.541ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59843
23902024-12-21T01:22:21.541ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
23912024-12-21T01:22:21.578ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
23922024-12-21T01:22:21.633ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91800 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59843 req_id = 515cd74e-9efd-43f7-82ca-a9eaf08e5a1b response_code = 204 uri = /switch-ports
23932024-12-21T01:22:21.635ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59843 req_id = cbca818e-082f-4cbb-9f05-c9c11b2e865a response_code = 200 uri = /network-bootstore-config
23942024-12-21T01:22:22.562ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49672
23952024-12-21T01:22:22.562ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49672 req_id = 22c7c45b-8fda-4423-8148-b07c85056435 response_code = 200 uri = /vpc-routes
23962024-12-21T01:22:22.573ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52938
23972024-12-21T01:22:22.573ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52938 req_id = e878999d-fee1-4d17-a7b4-4b1cd427c002 response_code = 200 uri = /vpc-routes
23982024-12-21T01:22:22.597ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
23992024-12-21T01:22:22.597ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 128 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52938 req_id = 15a97957-ad96-453b-ae71-ff5f0d417a5b response_code = 204 uri = /eip-gateways
24002024-12-21T01:22:22.608ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 326 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:52938 req_id = 980b37bc-7bef-46e8-b7f1-9b87b633a919 response_code = 204 uri = /vpc-routes
24012024-12-21T01:22:22.611ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
24022024-12-21T01:22:22.611ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:49672 req_id = 985e0abb-981f-4aff-b73e-0da93f038be4 response_code = 204 uri = /eip-gateways
24032024-12-21T01:22:22.623ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 245 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:49672 req_id = 48ead42c-036c-4f8f-bdaa-0f762faec243 response_code = 204 uri = /vpc-routes
24042024-12-21T01:22:22.677ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33003
24052024-12-21T01:22:22.678ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33003 req_id = 9023aca3-db10-430f-9589-8d40523ecbc9 response_code = 200 uri = /v2p
24062024-12-21T01:22:22.700ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50057
24072024-12-21T01:22:22.701ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50057 req_id = 5bb1a46a-0b35-48db-bc31-001f00240e1d response_code = 200 uri = /v2p
24082024-12-21T01:22:22.720ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47316
24092024-12-21T01:22:22.720ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
24102024-12-21T01:22:22.761ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
24112024-12-21T01:22:22.817ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96493 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:47316 req_id = 87bdde91-dc7a-49fd-bdd6-eb1fa7dd5527 response_code = 204 uri = /switch-ports
24122024-12-21T01:22:22.817ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63727
24132024-12-21T01:22:22.817ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
24142024-12-21T01:22:22.820ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3301 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47316 req_id = f13b1659-6e26-4534-b474-499f3422dbdf response_code = 200 uri = /network-bootstore-config
24152024-12-21T01:22:22.852ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
24162024-12-21T01:22:22.908ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91324 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:63727 req_id = 1fb6a295-b8d7-46e6-9108-794a5b3cf3be response_code = 204 uri = /switch-ports
24172024-12-21T01:22:22.909ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 147 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63727 req_id = d2e95211-bf93-44fa-8ef7-243978bb26f8 response_code = 200 uri = /network-bootstore-config
24182024-12-21T01:22:36.775ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:55463
24192024-12-21T01:22:36.775ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:55463 req_id = a31e55ee-69df-4063-870e-f10298adfd87 response_code = 200 uri = /baseboard
24202024-12-21T01:22:43.391ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7568 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = 3a19bdf5-591d-4647-be51-62f22b658075 response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
24212024-12-21T01:22:51.155ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50992
24222024-12-21T01:22:51.155ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50992 req_id = f1467be0-1ff3-43bf-9e9b-9e95cc088aad response_code = 200 uri = /vpc-routes
24232024-12-21T01:22:51.205ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
24242024-12-21T01:22:51.205ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50992 req_id = 2bc8bdd7-435c-4507-8f67-e43d26e77f7e response_code = 204 uri = /eip-gateways
24252024-12-21T01:22:51.216ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 329 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50992 req_id = be1b1fcf-464a-4534-8ab3-f87229cddef3 response_code = 204 uri = /vpc-routes
24262024-12-21T01:22:51.384ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41629
24272024-12-21T01:22:51.384ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41629 req_id = 9dae58d3-4692-4b15-a1db-4dbd72072069 response_code = 200 uri = /v2p
24282024-12-21T01:22:51.453ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58941
24292024-12-21T01:22:51.453ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
24302024-12-21T01:22:51.494ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
24312024-12-21T01:22:51.556ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103177 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:58941 req_id = ef3be5ea-4ab3-4dc8-b75d-a0a9a67f7f09 response_code = 204 uri = /switch-ports
24322024-12-21T01:22:51.560ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58941 req_id = bf360d02-400d-4f96-8ae0-0f230d637eae response_code = 200 uri = /network-bootstore-config
24332024-12-21T01:22:52.401ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49241
24342024-12-21T01:22:52.401ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49241 req_id = 8ff485b8-5ae3-4482-9be7-0ad76ace6225 response_code = 200 uri = /vpc-routes
24352024-12-21T01:22:52.414ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37326
24362024-12-21T01:22:52.414ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37326 req_id = 616bfcb7-c38e-49e2-a349-b7f309b5b373 response_code = 200 uri = /vpc-routes
24372024-12-21T01:22:52.429ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
24382024-12-21T01:22:52.429ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37326 req_id = cbdab8d1-b5b6-437b-9cc3-246db6fad404 response_code = 204 uri = /eip-gateways
24392024-12-21T01:22:52.439ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 280 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37326 req_id = 0420b0bd-0066-4cdb-b9bc-54833acf896e response_code = 204 uri = /vpc-routes
24402024-12-21T01:22:52.445ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
24412024-12-21T01:22:52.445ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:49241 req_id = 9cfa3c62-2e6e-4fd4-b7ac-047b3edd9f45 response_code = 204 uri = /eip-gateways
24422024-12-21T01:22:52.455ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 260 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:49241 req_id = 39bd1373-1b75-4f29-8e98-33745ed2f035 response_code = 204 uri = /vpc-routes
24432024-12-21T01:22:52.461ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54433
24442024-12-21T01:22:52.461ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
24452024-12-21T01:22:52.504ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
24462024-12-21T01:22:52.560ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99210 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:54433 req_id = 53bba339-a54c-4c2c-95f2-9f0d6ad085f1 response_code = 204 uri = /switch-ports
24472024-12-21T01:22:52.560ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54941
24482024-12-21T01:22:52.560ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34818
24492024-12-21T01:22:52.561ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 206 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34818 req_id = 6fd59e24-990b-44ac-9b5a-8fcba9855a6c response_code = 200 uri = /v2p
24502024-12-21T01:22:52.561ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 233 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54433 req_id = e26dbe2c-019d-4d5c-885e-14f7d8efb618 response_code = 200 uri = /network-bootstore-config
24512024-12-21T01:22:52.561ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 204 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54941 req_id = 5ddd2a07-c092-4c6b-b0c5-653ace7e314a response_code = 200 uri = /v2p
24522024-12-21T01:22:52.587ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62047
24532024-12-21T01:22:52.587ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
24542024-12-21T01:22:52.626ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
24552024-12-21T01:22:52.680ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93587 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:62047 req_id = 05dec0a3-003a-4c72-9705-c6a22332bf42 response_code = 204 uri = /switch-ports
24562024-12-21T01:22:52.681ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62047 req_id = 6cf519ff-2b2e-4afc-bfb9-f35775269946 response_code = 200 uri = /network-bootstore-config
24572024-12-21T01:23:06.807ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:41851
24582024-12-21T01:23:06.808ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 48 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:41851 req_id = c64689f1-d36a-4404-9792-1cea46e8f206 response_code = 200 uri = /baseboard
24592024-12-21T01:23:13.391ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7623 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = 791b7474-ebcb-45eb-a4c4-f058212b5224 response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
24602024-12-21T01:23:21.249ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42288
24612024-12-21T01:23:21.249ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42288 req_id = 8cd0cea2-ebf5-4224-b71c-b87066e1e68d response_code = 200 uri = /vpc-routes
24622024-12-21T01:23:21.310ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
24632024-12-21T01:23:21.310ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 77 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42288 req_id = 3b465d3a-df5f-4af8-a5f1-b7d3febfce13 response_code = 204 uri = /eip-gateways
24642024-12-21T01:23:21.335ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 266 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42288 req_id = 927326f1-a84b-4664-99b4-e007133af7d3 response_code = 204 uri = /vpc-routes
24652024-12-21T01:23:21.475ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44455
24662024-12-21T01:23:21.475ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44455 req_id = 7c9f4000-9d56-4898-bff8-c13aa293988a response_code = 200 uri = /v2p
24672024-12-21T01:23:21.489ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49449
24682024-12-21T01:23:21.489ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
24692024-12-21T01:23:21.526ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
24702024-12-21T01:23:21.583ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93201 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:49449 req_id = 9f88f8aa-9513-4fe7-9d66-6b055eb717bb response_code = 204 uri = /switch-ports
24712024-12-21T01:23:21.586ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49449 req_id = 60fedd64-1997-4ab2-9076-28cdb5db51ec response_code = 200 uri = /network-bootstore-config
24722024-12-21T01:23:22.531ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53623
24732024-12-21T01:23:22.531ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53623 req_id = 38c5c963-25c3-4614-a60f-cf72d8040aa3 response_code = 200 uri = /vpc-routes
24742024-12-21T01:23:22.586ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
24752024-12-21T01:23:22.586ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 109 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53623 req_id = 688a75c9-9412-4a76-91e9-a15c4f3e1406 response_code = 204 uri = /eip-gateways
24762024-12-21T01:23:22.599ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 298 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53623 req_id = 480bab25-7ef8-4b15-8ec9-e75b2da96586 response_code = 204 uri = /vpc-routes
24772024-12-21T01:23:22.616ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38411
24782024-12-21T01:23:22.616ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38411 req_id = 05dc2eaf-7a91-4b14-b91c-99ea9bf1bc8d response_code = 200 uri = /vpc-routes
24792024-12-21T01:23:22.623ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63325
24802024-12-21T01:23:22.623ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63325 req_id = bfc25b27-9716-45a7-8f70-d2185e40818c response_code = 200 uri = /v2p
24812024-12-21T01:23:22.669ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
24822024-12-21T01:23:22.669ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 66 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:38411 req_id = 29e2cf6a-bb64-4959-ac0c-1c7c65c1c3d9 response_code = 204 uri = /eip-gateways
24832024-12-21T01:23:22.679ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 236 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:38411 req_id = fab6ed48-2c3a-42a8-9e7a-7812421e098e response_code = 204 uri = /vpc-routes
24842024-12-21T01:23:22.707ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59880
24852024-12-21T01:23:22.707ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
24862024-12-21T01:23:22.745ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
24872024-12-21T01:23:22.800ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92549 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:59880 req_id = eb7bfd5c-818c-4d59-baee-9deedf44630c response_code = 204 uri = /switch-ports
24882024-12-21T01:23:22.800ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36152
24892024-12-21T01:23:22.800ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36152 req_id = af5da1f8-60ba-4d24-9d12-18b7133ac2e9 response_code = 200 uri = /v2p
24902024-12-21T01:23:22.800ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59880 req_id = 0d526acd-8ef0-47d4-8f34-ad405d320d95 response_code = 200 uri = /network-bootstore-config
24912024-12-21T01:23:22.817ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50025
24922024-12-21T01:23:22.817ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
24932024-12-21T01:23:22.855ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
24942024-12-21T01:23:22.911ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93832 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:50025 req_id = 3ae83f69-6bfe-433e-957c-fd4f00bbedee response_code = 204 uri = /switch-ports
24952024-12-21T01:23:22.911ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 190 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50025 req_id = 7a112100-0207-4717-88d7-bdbdc6ab4937 response_code = 200 uri = /network-bootstore-config
24962024-12-21T01:23:36.842ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:42896
24972024-12-21T01:23:36.842ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 36 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:42896 req_id = 993046ad-beb8-41b0-aed9-2ab02726fea2 response_code = 200 uri = /baseboard
24982024-12-21T01:23:43.391ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7320 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = c5dbd75e-b784-464a-a9cb-424296e29dce response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
24992024-12-21T01:23:51.218ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55986
25002024-12-21T01:23:51.219ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55986 req_id = 08b4a3be-4872-4ffa-8eb0-a8771e7eb157 response_code = 200 uri = /vpc-routes
25012024-12-21T01:23:51.259ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
25022024-12-21T01:23:51.259ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55986 req_id = 156ac64a-f9ce-4625-9c19-ae09f80fb66a response_code = 204 uri = /eip-gateways
25032024-12-21T01:23:51.269ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 254 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55986 req_id = 78b5eafd-3740-46d6-b410-ed2b85f3628e response_code = 204 uri = /vpc-routes
25042024-12-21T01:23:51.311ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57638
25052024-12-21T01:23:51.312ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57638 req_id = 8d1ec2fc-e439-4ae7-b8b7-a049bb8d93e1 response_code = 200 uri = /v2p
25062024-12-21T01:23:51.324ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33415
25072024-12-21T01:23:51.324ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
25082024-12-21T01:23:51.363ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
25092024-12-21T01:23:51.418ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93599 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:33415 req_id = 4e5cee1f-66be-4f9f-9428-dcf17327a7c0 response_code = 204 uri = /switch-ports
25102024-12-21T01:23:51.418ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33415 req_id = d89bdb0a-04e9-47d9-8b58-0ac337ccc0a9 response_code = 200 uri = /network-bootstore-config
25112024-12-21T01:23:52.359ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64398
25122024-12-21T01:23:52.360ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 52 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64398 req_id = 98587139-8f4c-4f99-b734-316c81e720ce response_code = 200 uri = /vpc-routes
25132024-12-21T01:23:52.394ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47532
25142024-12-21T01:23:52.394ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47532 req_id = ce427f56-3426-4cdb-a2c4-c540f25db905 response_code = 200 uri = /vpc-routes
25152024-12-21T01:23:52.412ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
25162024-12-21T01:23:52.413ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64398 req_id = e69e0e3d-bd16-4b6b-af2e-f129504623ba response_code = 204 uri = /eip-gateways
25172024-12-21T01:23:52.418ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
25182024-12-21T01:23:52.418ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 52 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47532 req_id = 6baa0406-3a7c-4bf3-9d46-1b8e9ba2576d response_code = 204 uri = /eip-gateways
25192024-12-21T01:23:52.423ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 257 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64398 req_id = 00b54287-219f-4bd8-bfee-3a3a6b42bab6 response_code = 204 uri = /vpc-routes
25202024-12-21T01:23:52.428ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 212 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47532 req_id = 60ecbe78-8a9f-453f-b940-4ccec5ff13d5 response_code = 204 uri = /vpc-routes
25212024-12-21T01:23:52.447ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55759
25222024-12-21T01:23:52.447ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
25232024-12-21T01:23:52.498ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
25242024-12-21T01:23:52.577ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 129822 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:55759 req_id = e06552d3-2266-4ae5-91a2-0efde087fd85 response_code = 204 uri = /switch-ports
25252024-12-21T01:23:52.577ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42923
25262024-12-21T01:23:52.577ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58539
25272024-12-21T01:23:52.577ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 252 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55759 req_id = 1fcbf947-d6a2-4875-9502-6770fe64d462 response_code = 200 uri = /network-bootstore-config
25282024-12-21T01:23:52.577ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 180 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42923 req_id = b6e4f900-c6ca-4808-a215-ab077040c1b8 response_code = 200 uri = /v2p
25292024-12-21T01:23:52.577ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 235 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58539 req_id = e5afad11-8e98-4b69-936f-5d695d8e1df9 response_code = 200 uri = /v2p
25302024-12-21T01:23:52.594ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55794
25312024-12-21T01:23:52.594ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
25322024-12-21T01:23:52.633ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
25332024-12-21T01:23:52.694ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100418 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:55794 req_id = ec65ba44-fb8e-458f-b240-cdb5f8043d11 response_code = 204 uri = /switch-ports
25342024-12-21T01:23:52.695ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 303 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55794 req_id = fac31ab0-9220-4563-852a-e7e9f49cd656 response_code = 200 uri = /network-bootstore-config
25352024-12-21T01:24:06.876ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:59548
25362024-12-21T01:24:06.876ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 32 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:59548 req_id = b4ca34ca-a5c0-4fd1-b999-aef8f3093f07 response_code = 200 uri = /baseboard
25372024-12-21T01:24:08.310ZINFOSledAgent (auto-cleanup-task): running automatic periodic zone bundle cleanup file = sled-agent/src/zone_bundle.rs:197
25382024-12-21T01:24:08.310ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
25392024-12-21T01:24:08.340ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 1935, tv_nsec: 222799115 } time_to_next_cleanup = 599.99999614s
25402024-12-21T01:24:13.391ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7616 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = b201f858-a624-4275-afc1-4ba0dd8db891 response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
25412024-12-21T01:24:21.299ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37496
25422024-12-21T01:24:21.300ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37496 req_id = 8b58e03a-0415-48fc-8bd1-91c891a8ec45 response_code = 200 uri = /vpc-routes
25432024-12-21T01:24:21.315ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
25442024-12-21T01:24:21.315ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:37496 req_id = 2ac632c9-a826-434c-9325-0ba3c0cf7d2e response_code = 204 uri = /eip-gateways
25452024-12-21T01:24:21.323ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 231 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:37496 req_id = d4c08e7b-fa46-417d-b716-3595b3d3e81d response_code = 204 uri = /vpc-routes
25462024-12-21T01:24:21.485ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46618
25472024-12-21T01:24:21.485ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46618 req_id = c15c28ce-0a2d-4fb8-91c9-fc4d25e5fd7b response_code = 200 uri = /v2p
25482024-12-21T01:24:21.568ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62816
25492024-12-21T01:24:21.569ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
25502024-12-21T01:24:21.607ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
25512024-12-21T01:24:21.659ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90831 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:62816 req_id = 6e3378a3-920f-460e-aa80-240606dde3d2 response_code = 204 uri = /switch-ports
25522024-12-21T01:24:21.664ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62816 req_id = f49a1b30-19d3-4e6e-a4a9-cbc77d5d0586 response_code = 200 uri = /network-bootstore-config
25532024-12-21T01:24:22.430ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
25542024-12-21T01:24:22.447ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50481
25552024-12-21T01:24:22.447ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50481 req_id = 20f64274-5f61-424e-a41a-28e1fd6b9462 response_code = 200 uri = /vpc-routes
25562024-12-21T01:24:22.469ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47903
25572024-12-21T01:24:22.470ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47903 req_id = 8698e106-4455-4d58-adb9-35ac61b6301e response_code = 200 uri = /vpc-routes
25582024-12-21T01:24:22.515ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
25592024-12-21T01:24:22.515ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:50481 req_id = 9c59eeca-790d-47ad-861d-50112bf7c8d0 response_code = 204 uri = /eip-gateways
25602024-12-21T01:24:22.525ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 202 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:50481 req_id = 69999f9c-6340-4082-9f78-35420c0d679f response_code = 204 uri = /vpc-routes
25612024-12-21T01:24:22.536ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
25622024-12-21T01:24:22.536ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47903 req_id = 68b191ba-1ad7-403a-961a-ae53bf59e91e response_code = 204 uri = /eip-gateways
25632024-12-21T01:24:22.546ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 156 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47903 req_id = 8ba99734-350e-41bf-8a89-61f226c2b553 response_code = 204 uri = /vpc-routes
25642024-12-21T01:24:22.615ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38876
25652024-12-21T01:24:22.615ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 140 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38876 req_id = 9f104c49-9f5d-4153-b289-6a40c7aee0df response_code = 200 uri = /v2p
25662024-12-21T01:24:22.642ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46788
25672024-12-21T01:24:22.642ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
25682024-12-21T01:24:22.684ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
25692024-12-21T01:24:22.747ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105371 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:46788 req_id = 06369b52-03f7-4e6a-9cfb-8bbf59b93e2c response_code = 204 uri = /switch-ports
25702024-12-21T01:24:22.748ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46788 req_id = b27977a2-f8dd-48ee-af04-b2b5b5528e0c response_code = 200 uri = /network-bootstore-config
25712024-12-21T01:24:22.779ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37328
25722024-12-21T01:24:22.779ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 158 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37328 req_id = 3866b06a-8010-4446-93ff-207a19ce3446 response_code = 200 uri = /v2p
25732024-12-21T01:24:22.873ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49506
25742024-12-21T01:24:22.874ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
25752024-12-21T01:24:22.911ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
25762024-12-21T01:24:22.968ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93995 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:49506 req_id = 2095d841-311c-496a-a6df-7a60e87d0e95 response_code = 204 uri = /switch-ports
25772024-12-21T01:24:22.968ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49506 req_id = b1f1d10e-174b-40ae-9686-271ce428d9ba response_code = 200 uri = /network-bootstore-config
25782024-12-21T01:24:36.911ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:43665
25792024-12-21T01:24:36.912ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:43665 req_id = d32d5072-df70-40fc-83cf-ff18b3f2a20d response_code = 200 uri = /baseboard
25802024-12-21T01:24:43.392ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7909 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = e5f6a456-6378-49ff-91be-796bdb3a0489 response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
25812024-12-21T01:24:51.235ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56140
25822024-12-21T01:24:51.235ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56140 req_id = 6e3324d2-e423-442c-bbb6-6088709d938e response_code = 200 uri = /vpc-routes
25832024-12-21T01:24:51.281ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
25842024-12-21T01:24:51.281ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 66 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56140 req_id = a2407201-d1da-45b3-8ac6-9c3e3b3f5941 response_code = 204 uri = /eip-gateways
25852024-12-21T01:24:51.315ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50333
25862024-12-21T01:24:51.315ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50333 req_id = 0af8bc14-f714-44d2-8965-aba1ba78aa52 response_code = 200 uri = /v2p
25872024-12-21T01:24:51.322ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 206 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:56140 req_id = 3240a937-8214-40a0-8a8b-2b708b7a2312 response_code = 204 uri = /vpc-routes
25882024-12-21T01:24:51.357ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39492
25892024-12-21T01:24:51.357ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
25902024-12-21T01:24:51.398ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
25912024-12-21T01:24:51.458ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100910 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:39492 req_id = 43e6ff75-5ba8-4048-aa4b-80df7381574a response_code = 204 uri = /switch-ports
25922024-12-21T01:24:51.458ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39492 req_id = b57b867d-cec8-42c9-82f8-a6f65f00c40a response_code = 200 uri = /network-bootstore-config
25932024-12-21T01:24:52.284ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47374
25942024-12-21T01:24:52.284ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47374 req_id = 88b36e6c-68fc-4a10-b140-42ff3435403d response_code = 200 uri = /vpc-routes
25952024-12-21T01:24:52.338ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
25962024-12-21T01:24:52.338ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:47374 req_id = 572e2191-40db-4750-8e3f-b3aa19926bfa response_code = 204 uri = /eip-gateways
25972024-12-21T01:24:52.347ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 263 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:47374 req_id = 57570a25-72fe-4e70-9c10-d3dfad50317e response_code = 204 uri = /vpc-routes
25982024-12-21T01:24:52.423ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52859
25992024-12-21T01:24:52.423ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52859 req_id = 8820a0d7-9b44-429a-ba68-7a0e0efd2a5b response_code = 200 uri = /vpc-routes
26002024-12-21T01:24:52.444ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:59984
26012024-12-21T01:24:52.444ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:59984 req_id = a369ee65-c2a1-42ef-8ce0-407427ce3551 response_code = 200 uri = /v2p
26022024-12-21T01:24:52.471ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
26032024-12-21T01:24:52.471ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52859 req_id = 4cfe2632-9f77-46df-81cf-f61147cb2d15 response_code = 204 uri = /eip-gateways
26042024-12-21T01:24:52.480ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 174 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:52859 req_id = 705da4e5-53e3-4493-bc34-94f359c16792 response_code = 204 uri = /vpc-routes
26052024-12-21T01:24:52.483ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37903
26062024-12-21T01:24:52.484ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
26072024-12-21T01:24:52.523ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
26082024-12-21T01:24:52.580ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96486 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:37903 req_id = 645221e8-ad70-4131-92d1-1a1d450d3cb3 response_code = 204 uri = /switch-ports
26092024-12-21T01:24:52.580ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58616
26102024-12-21T01:24:52.580ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57843
26112024-12-21T01:24:52.580ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 246 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37903 req_id = c7316dd0-6160-4bc2-ac30-81286f31aabf response_code = 200 uri = /network-bootstore-config
26122024-12-21T01:24:52.580ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 189 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57843 req_id = a6532eb4-d527-4e33-90c4-265ea8fa30c2 response_code = 200 uri = /v2p
26132024-12-21T01:24:52.581ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
26142024-12-21T01:24:52.616ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
26152024-12-21T01:24:52.670ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89560 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:58616 req_id = 365c6bb7-0e79-4f9d-a300-f6e2d2bb53fb response_code = 204 uri = /switch-ports
26162024-12-21T01:24:52.670ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 150 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58616 req_id = d272aa90-278d-4b89-88b2-c8d63d43705e response_code = 200 uri = /network-bootstore-config
26172024-12-21T01:25:06.946ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:61411
26182024-12-21T01:25:06.946ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:61411 req_id = 58d5fcff-ac71-4d92-8b3a-23092354a906 response_code = 200 uri = /baseboard
26192024-12-21T01:25:13.392ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8256 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = d21e6ab4-3b2f-4930-863e-03e8cff517db response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
26202024-12-21T01:25:21.316ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35272
26212024-12-21T01:25:21.316ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35272 req_id = 3ce170d4-7af5-4995-9ed7-d7f6331bec81 response_code = 200 uri = /vpc-routes
26222024-12-21T01:25:21.363ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
26232024-12-21T01:25:21.363ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 66 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35272 req_id = a265c5dc-4cbe-4184-beaf-adf92a3beb3e response_code = 204 uri = /eip-gateways
26242024-12-21T01:25:21.480ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57701
26252024-12-21T01:25:21.480ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57701 req_id = 5e392b96-2319-4b63-af2a-1f36f23ab56f response_code = 200 uri = /v2p
26262024-12-21T01:25:21.485ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 232 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35272 req_id = 4971cb7d-08dd-4288-ab14-472b002f7ca6 response_code = 204 uri = /vpc-routes
26272024-12-21T01:25:21.640ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53141
26282024-12-21T01:25:21.641ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
26292024-12-21T01:25:21.679ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
26302024-12-21T01:25:21.735ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94271 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:53141 req_id = c3ab1940-452b-47e6-8599-156bafa0a19e response_code = 204 uri = /switch-ports
26312024-12-21T01:25:21.735ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 165 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53141 req_id = 822203e0-a0ca-4539-82d5-fb3dc351a23e response_code = 200 uri = /network-bootstore-config
26322024-12-21T01:25:22.391ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45303
26332024-12-21T01:25:22.391ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45303 req_id = 6b7caf38-f65f-4b90-8081-5dbbeef8d2b1 response_code = 200 uri = /vpc-routes
26342024-12-21T01:25:22.467ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
26352024-12-21T01:25:22.467ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:45303 req_id = ea47fc95-01e2-4339-b408-e8715ab840a4 response_code = 204 uri = /eip-gateways
26362024-12-21T01:25:22.478ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 252 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:45303 req_id = b8e1eb15-b571-454f-9e21-429f60a4340f response_code = 204 uri = /vpc-routes
26372024-12-21T01:25:22.617ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36779
26382024-12-21T01:25:22.617ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36779 req_id = d25abae8-3e45-4f95-b717-d17dfb315cb7 response_code = 200 uri = /v2p
26392024-12-21T01:25:22.654ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57667
26402024-12-21T01:25:22.654ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57667 req_id = 8f880a78-99ac-45b3-a7d2-38b7d17584df response_code = 200 uri = /vpc-routes
26412024-12-21T01:25:22.701ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
26422024-12-21T01:25:22.701ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57667 req_id = 4772a4cb-c75a-4457-a42d-bd32b6b9cec4 response_code = 204 uri = /eip-gateways
26432024-12-21T01:25:22.712ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 250 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:57667 req_id = 4e13ff6b-6b40-4e1c-bd0a-06dceede470f response_code = 204 uri = /vpc-routes
26442024-12-21T01:25:22.738ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44146
26452024-12-21T01:25:22.738ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
26462024-12-21T01:25:22.776ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
26472024-12-21T01:25:22.831ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92489 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:44146 req_id = b3911013-8242-46c6-b9c7-740b1749832a response_code = 204 uri = /switch-ports
26482024-12-21T01:25:22.831ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35155
26492024-12-21T01:25:22.831ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 206 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44146 req_id = d1564088-521d-4def-ae60-07886a30193e response_code = 200 uri = /network-bootstore-config
26502024-12-21T01:25:22.832ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 194 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35155 req_id = 243eca88-9fdb-4216-84b8-cbd669990b33 response_code = 200 uri = /v2p
26512024-12-21T01:25:22.890ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36891
26522024-12-21T01:25:22.890ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
26532024-12-21T01:25:22.924ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
26542024-12-21T01:25:22.981ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90932 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:36891 req_id = 7888a2f2-5dff-4609-97d6-95fc416e6e15 response_code = 204 uri = /switch-ports
26552024-12-21T01:25:22.981ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36891 req_id = 2b8b0271-4c56-44ea-81b0-7b0de999e602 response_code = 200 uri = /network-bootstore-config
26562024-12-21T01:25:36.979ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:38628
26572024-12-21T01:25:36.979ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:38628 req_id = f391ba6f-d4b0-4bd5-b396-f4df27ca2250 response_code = 200 uri = /baseboard
26582024-12-21T01:25:43.392ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7930 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = 965a8769-3b56-4e9d-924f-2a5effbe76d1 response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
26592024-12-21T01:25:51.215ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33318
26602024-12-21T01:25:51.215ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33318 req_id = 46233e91-6ee6-45f1-afbc-27de0da6ff32 response_code = 200 uri = /vpc-routes
26612024-12-21T01:25:51.221ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65175
26622024-12-21T01:25:51.221ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65175 req_id = 2545c4d9-4e94-4768-91da-b36a48bcd5db response_code = 200 uri = /v2p
26632024-12-21T01:25:51.230ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
26642024-12-21T01:25:51.230ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 52 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:33318 req_id = 90415e2f-7392-4abf-a304-e1bab06d9b66 response_code = 204 uri = /eip-gateways
26652024-12-21T01:25:51.239ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 247 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:33318 req_id = 8e3088b3-ba57-4749-96fc-fe290e77e658 response_code = 204 uri = /vpc-routes
26662024-12-21T01:25:51.304ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46463
26672024-12-21T01:25:51.304ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
26682024-12-21T01:25:51.343ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
26692024-12-21T01:25:51.398ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94001 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:46463 req_id = 086dc274-6d8e-4bde-ba8e-a1b8d536f378 response_code = 204 uri = /switch-ports
26702024-12-21T01:25:51.398ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46463 req_id = 0d3b3e06-98c3-4bb7-bf11-b28dafac71d5 response_code = 200 uri = /network-bootstore-config
26712024-12-21T01:25:52.399ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61240
26722024-12-21T01:25:52.399ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61240 req_id = e9a71fd1-7de5-4b6e-b41d-9b68cfd88e4d response_code = 200 uri = /vpc-routes
26732024-12-21T01:25:52.402ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33557
26742024-12-21T01:25:52.402ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 33 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33557 req_id = a1841223-596a-48c9-b0bf-acdb00831962 response_code = 200 uri = /vpc-routes
26752024-12-21T01:25:52.417ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
26762024-12-21T01:25:52.417ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 68 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:33557 req_id = 1bd87fd5-f624-408c-a701-e3eeffcab6ff response_code = 204 uri = /eip-gateways
26772024-12-21T01:25:52.446ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
26782024-12-21T01:25:52.446ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61240 req_id = 8a65ae1a-5671-4b57-8a3f-9c6553852b05 response_code = 204 uri = /eip-gateways
26792024-12-21T01:25:52.464ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 233 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61240 req_id = a32df5ba-756f-4849-a15e-85c588ffc18f response_code = 204 uri = /vpc-routes
26802024-12-21T01:25:52.494ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56900
26812024-12-21T01:25:52.495ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 148 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56900 req_id = 5ced0fc6-6c7b-4c1f-9d8f-8d828631e8a4 response_code = 200 uri = /v2p
26822024-12-21T01:25:52.498ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:33557 req_id = 482d6c06-9084-4053-8159-934f2770d3b5 response_code = 204 uri = /vpc-routes
26832024-12-21T01:25:52.502ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44363
26842024-12-21T01:25:52.503ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44363 req_id = 1a35d4b7-7947-4aed-9e4b-615d017d1b35 response_code = 200 uri = /v2p
26852024-12-21T01:25:52.518ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37840
26862024-12-21T01:25:52.518ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
26872024-12-21T01:25:52.556ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
26882024-12-21T01:25:52.614ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95787 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:37840 req_id = 4ce5f129-229c-408c-a64b-cfeeaf8c8db8 response_code = 204 uri = /switch-ports
26892024-12-21T01:25:52.614ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51153
26902024-12-21T01:25:52.614ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
26912024-12-21T01:25:52.618ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3600 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37840 req_id = 012b873a-feef-4cf4-93e8-e1c194219b25 response_code = 200 uri = /network-bootstore-config
26922024-12-21T01:25:52.649ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
26932024-12-21T01:25:52.706ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92541 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:51153 req_id = f8a9b14f-c71f-4a04-8361-2b8c3fc088cd response_code = 204 uri = /switch-ports
26942024-12-21T01:25:52.707ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 156 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51153 req_id = cc70f2dc-b6b5-4499-83c4-bc9bcead346a response_code = 200 uri = /network-bootstore-config
26952024-12-21T01:26:07.013ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:59445
26962024-12-21T01:26:07.013ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 63 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:59445 req_id = 9bec753f-e05c-4afc-9d2a-69672c71a0ed response_code = 200 uri = /baseboard
26972024-12-21T01:26:13.392ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7816 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = db707107-8fd1-4f00-a68c-5f39d71d0c6c response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
26982024-12-21T01:26:21.483ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58251
26992024-12-21T01:26:21.483ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 46 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58251 req_id = 8394383d-822f-4f07-9edc-6b7b2de802e1 response_code = 200 uri = /vpc-routes
27002024-12-21T01:26:21.508ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
27012024-12-21T01:26:21.508ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58251 req_id = 98df10af-e662-4393-ad3e-daee7862ef6d response_code = 204 uri = /eip-gateways
27022024-12-21T01:26:21.518ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 278 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58251 req_id = f3e8a98e-45dc-4143-a820-6cb6df8ae2f9 response_code = 204 uri = /vpc-routes
27032024-12-21T01:26:21.635ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41157
27042024-12-21T01:26:21.636ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
27052024-12-21T01:26:21.636ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
27062024-12-21T01:26:21.636ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-zones.json file = common/src/ledger.rs:177
27072024-12-21T01:26:21.636ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-zones.json file = common/src/ledger.rs:177
27082024-12-21T01:26:21.642ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60809
27092024-12-21T01:26:21.642ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 296 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60809 req_id = 3c74fd2c-5652-488f-9d67-51e3184ecd01 response_code = 200 uri = /v2p
27102024-12-21T01:26:21.649ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27112024-12-21T01:26:21.649ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff
27122024-12-21T01:26:21.655ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57900
27132024-12-21T01:26:21.655ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
27142024-12-21T01:26:21.678ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27152024-12-21T01:26:21.678ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_41cc83ed-adbe-492f-ae31-7074b31692b0
27162024-12-21T01:26:21.698ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
27172024-12-21T01:26:21.707ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27182024-12-21T01:26:21.707ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_7e02d0c4-4e5d-440d-94c1-593a67af3118
27192024-12-21T01:26:21.734ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27202024-12-21T01:26:21.734ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50
27212024-12-21T01:26:21.758ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27222024-12-21T01:26:21.758ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_10325d85-ce1b-49aa-8039-7e9482ec8803
27232024-12-21T01:26:21.764ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 109074 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:57900 req_id = aac3693b-15cb-4ff4-8b93-2def83e0443b response_code = 204 uri = /switch-ports
27242024-12-21T01:26:21.764ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57900 req_id = 9eb9c93b-b0db-4e6b-bace-446f6e5d6463 response_code = 200 uri = /network-bootstore-config
27252024-12-21T01:26:21.782ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27262024-12-21T01:26:21.782ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739
27272024-12-21T01:26:21.792ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55713
27282024-12-21T01:26:21.793ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 44a38c79-185d-44b1-a294-e3a088dd4f4f, kind: Service { id: c4364b0f-0b76-413b-9126-391216a47153 }, name: Name("external-dns-c4364b0f-0b76-413b-9126-391216a47153"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cd175b09-5785-415d-8d32-a617204393a9, kind: Service { id: 24428c69-128b-4995-bc7a-addc66912490 }, name: Name("external-dns-24428c69-128b-4995-bc7a-addc66912490"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 190])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 05db974b-0405-4131-ae02-a37152ea980b, kind: Service { id: fe9c2ca3-0266-4081-952b-4cb3f2bf1371 }, name: Name("nexus-fe9c2ca3-0266-4081-952b-4cb3f2bf1371"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 44])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5b5216f1-9871-4dfb-aced-f4b109e9a1c2, kind: Service { id: ee35abf0-6140-47bd-8a0f-da60e1b0cdad }, name: Name("nexus-ee35abf0-6140-47bd-8a0f-da60e1b0cdad"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 654f7fda-6e14-4220-bc54-c5c6d8426d7e, kind: Service { id: bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1 }, name: Name("nexus-bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 234, 194])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
27292024-12-21T01:26:21.793ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27302024-12-21T01:26:21.793ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27312024-12-21T01:26:21.793ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27322024-12-21T01:26:21.793ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27332024-12-21T01:26:21.793ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27342024-12-21T01:26:21.793ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
27352024-12-21T01:26:21.793ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 478 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55713 req_id = 65f76820-38f3-4d39-b9a0-290509815b43 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
27362024-12-21T01:26:21.804ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27372024-12-21T01:26:21.804ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_192cce10-69a8-4ed6-a459-9ca86a7206ec
27382024-12-21T01:26:21.826ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27392024-12-21T01:26:21.826ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_34789575-52b6-411b-9614-90f11629cb04
27402024-12-21T01:26:21.848ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27412024-12-21T01:26:21.850ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_e80b2509-2264-41c1-be0f-a3173a243a47
27422024-12-21T01:26:21.860ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 224092 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41157 req_id = d6f34e64-52e1-4dbb-9ad4-9ab52b1db9cd response_code = 200 uri = /inventory
27432024-12-21T01:26:22.525ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64553
27442024-12-21T01:26:22.525ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64553 req_id = cbb99bd0-df2f-48cb-a3aa-76c8747753e9 response_code = 200 uri = /vpc-routes
27452024-12-21T01:26:22.580ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
27462024-12-21T01:26:22.580ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64553 req_id = cf442459-d2aa-4f0e-9f01-f27e00e50714 response_code = 204 uri = /eip-gateways
27472024-12-21T01:26:22.595ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 352 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64553 req_id = 6b110f2e-dfe9-4bfd-924d-232e5d104886 response_code = 204 uri = /vpc-routes
27482024-12-21T01:26:22.699ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:65279
27492024-12-21T01:26:22.699ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
27502024-12-21T01:26:22.699ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
27512024-12-21T01:26:22.699ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-zones.json file = common/src/ledger.rs:177
27522024-12-21T01:26:22.699ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-zones.json file = common/src/ledger.rs:177
27532024-12-21T01:26:22.706ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58970
27542024-12-21T01:26:22.706ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63745
27552024-12-21T01:26:22.707ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 296 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58970 req_id = c649fb64-2c3b-4769-a136-d20f4dc7adf1 response_code = 200 uri = /vpc-routes
27562024-12-21T01:26:22.707ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
27572024-12-21T01:26:22.717ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27582024-12-21T01:26:22.717ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff
27592024-12-21T01:26:22.760ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27602024-12-21T01:26:22.760ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_41cc83ed-adbe-492f-ae31-7074b31692b0
27612024-12-21T01:26:22.767ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
27622024-12-21T01:26:22.775ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
27632024-12-21T01:26:22.775ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 359 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58970 req_id = 949c266d-2618-444b-be98-cd47dc284282 response_code = 204 uri = /eip-gateways
27642024-12-21T01:26:22.788ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 950 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:58970 req_id = e4dcb235-f7e5-4c72-a0c0-16f3837250b1 response_code = 204 uri = /vpc-routes
27652024-12-21T01:26:22.800ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27662024-12-21T01:26:22.800ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_7e02d0c4-4e5d-440d-94c1-593a67af3118
27672024-12-21T01:26:22.834ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46930
27682024-12-21T01:26:22.834ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 190 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46930 req_id = 25911eaa-c7e4-4455-a6ea-44a8e27724a7 response_code = 200 uri = /v2p
27692024-12-21T01:26:22.838ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27702024-12-21T01:26:22.838ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50
27712024-12-21T01:26:22.862ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 155316 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:63745 req_id = b60277ba-219b-40e8-b682-ae635f656ef2 response_code = 204 uri = /switch-ports
27722024-12-21T01:26:22.862ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:65096
27732024-12-21T01:26:22.862ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46139
27742024-12-21T01:26:22.862ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 310 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63745 req_id = faadcd5e-46d2-4d88-99c4-863aad5ec2f8 response_code = 200 uri = /network-bootstore-config
27752024-12-21T01:26:22.862ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
27762024-12-21T01:26:22.862ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
27772024-12-21T01:26:22.863ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 340 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:65096 req_id = 249b5731-9d87-406a-b0cb-b9d31ce76314 response_code = 200 uri = /v2p
27782024-12-21T01:26:22.863ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/config/omicron-zones.json file = common/src/ledger.rs:177
27792024-12-21T01:26:22.867ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/config/omicron-zones.json file = common/src/ledger.rs:177
27802024-12-21T01:26:22.872ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27812024-12-21T01:26:22.873ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_10325d85-ce1b-49aa-8039-7e9482ec8803
27822024-12-21T01:26:22.879ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34592
27832024-12-21T01:26:22.880ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
27842024-12-21T01:26:22.891ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 3d4d0721-dd16-40d1-afff-d64c8f3a10ff (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
27852024-12-21T01:26:22.891ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_3d4d0721-dd16-40d1-afff-d64c8f3a10ff
27862024-12-21T01:26:22.900ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38885
27872024-12-21T01:26:22.901ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51048
27882024-12-21T01:26:22.901ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 44a38c79-185d-44b1-a294-e3a088dd4f4f, kind: Service { id: c4364b0f-0b76-413b-9126-391216a47153 }, name: Name("external-dns-c4364b0f-0b76-413b-9126-391216a47153"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cd175b09-5785-415d-8d32-a617204393a9, kind: Service { id: 24428c69-128b-4995-bc7a-addc66912490 }, name: Name("external-dns-24428c69-128b-4995-bc7a-addc66912490"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 190])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 05db974b-0405-4131-ae02-a37152ea980b, kind: Service { id: fe9c2ca3-0266-4081-952b-4cb3f2bf1371 }, name: Name("nexus-fe9c2ca3-0266-4081-952b-4cb3f2bf1371"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 44])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5b5216f1-9871-4dfb-aced-f4b109e9a1c2, kind: Service { id: ee35abf0-6140-47bd-8a0f-da60e1b0cdad }, name: Name("nexus-ee35abf0-6140-47bd-8a0f-da60e1b0cdad"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 654f7fda-6e14-4220-bc54-c5c6d8426d7e, kind: Service { id: bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1 }, name: Name("nexus-bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 234, 194])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
27892024-12-21T01:26:22.901ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27902024-12-21T01:26:22.901ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27912024-12-21T01:26:22.902ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27922024-12-21T01:26:22.902ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27932024-12-21T01:26:22.902ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27942024-12-21T01:26:22.902ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
27952024-12-21T01:26:22.902ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1147 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:38885 req_id = 85de1cf2-c023-4c3a-aa6c-12a53fae7ee9 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
27962024-12-21T01:26:22.902ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 44a38c79-185d-44b1-a294-e3a088dd4f4f, kind: Service { id: c4364b0f-0b76-413b-9126-391216a47153 }, name: Name("external-dns-c4364b0f-0b76-413b-9126-391216a47153"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 240, 244])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: cd175b09-5785-415d-8d32-a617204393a9, kind: Service { id: 24428c69-128b-4995-bc7a-addc66912490 }, name: Name("external-dns-24428c69-128b-4995-bc7a-addc66912490"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 190])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 05db974b-0405-4131-ae02-a37152ea980b, kind: Service { id: fe9c2ca3-0266-4081-952b-4cb3f2bf1371 }, name: Name("nexus-fe9c2ca3-0266-4081-952b-4cb3f2bf1371"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 186, 44])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 5b5216f1-9871-4dfb-aced-f4b109e9a1c2, kind: Service { id: ee35abf0-6140-47bd-8a0f-da60e1b0cdad }, name: Name("nexus-ee35abf0-6140-47bd-8a0f-da60e1b0cdad"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 131, 56])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 654f7fda-6e14-4220-bc54-c5c6d8426d7e, kind: Service { id: bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1 }, name: Name("nexus-bb861d0b-0ac2-4c45-b09c-c62f0cfddaa1"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 234, 194])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
27972024-12-21T01:26:22.902ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27982024-12-21T01:26:22.902ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27992024-12-21T01:26:22.902ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
28002024-12-21T01:26:22.902ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
28012024-12-21T01:26:22.902ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
28022024-12-21T01:26:22.902ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
28032024-12-21T01:26:22.902ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 421 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51048 req_id = d49e5893-f780-4687-b605-382de443deb3 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
28042024-12-21T01:26:22.920ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 41cc83ed-adbe-492f-ae31-7074b31692b0 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28052024-12-21T01:26:22.921ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_41cc83ed-adbe-492f-ae31-7074b31692b0
28062024-12-21T01:26:22.931ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
28072024-12-21T01:26:22.937ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28082024-12-21T01:26:22.937ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739
28092024-12-21T01:26:22.956ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 7e02d0c4-4e5d-440d-94c1-593a67af3118 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28102024-12-21T01:26:22.956ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_7e02d0c4-4e5d-440d-94c1-593a67af3118
28112024-12-21T01:26:22.973ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28122024-12-21T01:26:22.973ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_192cce10-69a8-4ed6-a459-9ca86a7206ec
28132024-12-21T01:26:23.000ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28142024-12-21T01:26:23.000ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_34789575-52b6-411b-9614-90f11629cb04
28152024-12-21T01:26:23.005ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 125563 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:34592 req_id = 0e4fa9b4-cfcb-4824-a3d0-3196a087ae6a response_code = 204 uri = /switch-ports
28162024-12-21T01:26:23.006ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 303 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34592 req_id = 37de87dd-a812-4ad8-913a-9f754cc47ec7 response_code = 200 uri = /network-bootstore-config
28172024-12-21T01:26:23.016ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 22c20cb3-3659-48b7-b12d-b9ae21224e50 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28182024-12-21T01:26:23.016ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_22c20cb3-3659-48b7-b12d-b9ae21224e50
28192024-12-21T01:26:23.033ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28202024-12-21T01:26:23.033ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_e80b2509-2264-41c1-be0f-a3173a243a47
28212024-12-21T01:26:23.048ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 10325d85-ce1b-49aa-8039-7e9482ec8803 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28222024-12-21T01:26:23.048ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_10325d85-ce1b-49aa-8039-7e9482ec8803
28232024-12-21T01:26:23.067ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 368130 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:65279 req_id = e94b29b0-f71a-4e45-ac1f-378d5d244e7e response_code = 200 uri = /inventory
28242024-12-21T01:26:23.073ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28252024-12-21T01:26:23.073ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739
28262024-12-21T01:26:23.097ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 192cce10-69a8-4ed6-a459-9ca86a7206ec (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28272024-12-21T01:26:23.097ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_192cce10-69a8-4ed6-a459-9ca86a7206ec
28282024-12-21T01:26:23.123ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 34789575-52b6-411b-9614-90f11629cb04 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28292024-12-21T01:26:23.123ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_34789575-52b6-411b-9614-90f11629cb04
28302024-12-21T01:26:23.149ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: e80b2509-2264-41c1-be0f-a3173a243a47 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:607
28312024-12-21T01:26:23.149ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1118 request = datasets_list zpool = oxp_e80b2509-2264-41c1-be0f-a3173a243a47
28322024-12-21T01:26:23.163ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 301181 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46139 req_id = b6eaac8b-17c3-4adf-9391-c0c0e08f1e45 response_code = 200 uri = /inventory
28332024-12-21T01:26:37.047ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:53530
28342024-12-21T01:26:37.047ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:53530 req_id = 01b1cd89-c4bc-4f69-aa39-fc09b0962ee6 response_code = 200 uri = /baseboard
28352024-12-21T01:26:43.392ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7863 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = 971c5b8b-8f47-4e4f-b14c-4542163107d3 response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
28362024-12-21T01:26:51.249ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55344
28372024-12-21T01:26:51.249ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 73 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55344 req_id = 36f1f313-3ef9-423e-a92c-af7607f830b5 response_code = 200 uri = /vpc-routes
28382024-12-21T01:26:51.265ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
28392024-12-21T01:26:51.265ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 68 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55344 req_id = 2bfbfa38-2f18-478a-8668-e990d90c739e response_code = 204 uri = /eip-gateways
28402024-12-21T01:26:51.277ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 309 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55344 req_id = 2c2d4934-d036-4702-a63e-11128bc8f4bb response_code = 204 uri = /vpc-routes
28412024-12-21T01:26:51.329ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42285
28422024-12-21T01:26:51.329ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
28432024-12-21T01:26:51.369ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
28442024-12-21T01:26:51.431ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102604 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:42285 req_id = 94bb273a-a55e-4167-9d8d-28ee5d0d9305 response_code = 204 uri = /switch-ports
28452024-12-21T01:26:51.431ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58522
28462024-12-21T01:26:51.432ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 150 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42285 req_id = bb85e2f7-0602-43f9-8548-2bea243f97aa response_code = 200 uri = /network-bootstore-config
28472024-12-21T01:26:51.432ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 182 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58522 req_id = c95b8a1b-3777-4525-9751-a30de050f78a response_code = 200 uri = /v2p
28482024-12-21T01:26:52.346ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61921
28492024-12-21T01:26:52.346ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61921 req_id = d1e05c3f-93db-4c74-92a9-2c981ac2841a response_code = 200 uri = /vpc-routes
28502024-12-21T01:26:52.391ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
28512024-12-21T01:26:52.391ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61921 req_id = 250ace9d-dbf2-486e-a3a4-94a9ff6aa1e0 response_code = 204 uri = /eip-gateways
28522024-12-21T01:26:52.402ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 300 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61921 req_id = 95e07e18-dd57-4966-9b79-e789899587a6 response_code = 204 uri = /vpc-routes
28532024-12-21T01:26:52.418ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52057
28542024-12-21T01:26:52.418ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
28552024-12-21T01:26:52.461ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
28562024-12-21T01:26:52.516ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97864 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:52057 req_id = 29a92a3a-6b8a-4d64-afaa-37da13094b54 response_code = 204 uri = /switch-ports
28572024-12-21T01:26:52.516ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38341
28582024-12-21T01:26:52.516ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49910
28592024-12-21T01:26:52.516ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61555
28602024-12-21T01:26:52.516ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 232 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61555 req_id = a858c22d-57ac-4395-9d21-f9e435cf8be0 response_code = 200 uri = /v2p
28612024-12-21T01:26:52.516ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 345 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52057 req_id = b834f0d5-dbb1-4e2e-91b0-2666351350da response_code = 200 uri = /network-bootstore-config
28622024-12-21T01:26:52.516ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 251 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38341 req_id = 67a25fbf-834e-46d5-8122-fe1557a8de07 response_code = 200 uri = /v2p
28632024-12-21T01:26:52.516ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49910 req_id = 273494d4-958b-4764-8142-42001ca5c6bc response_code = 200 uri = /vpc-routes
28642024-12-21T01:26:52.547ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:35958
28652024-12-21T01:26:52.547ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
28662024-12-21T01:26:52.585ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
28672024-12-21T01:26:52.642ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95090 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:35958 req_id = c67b8558-36ac-40dd-9512-81fdc25f6572 response_code = 204 uri = /switch-ports
28682024-12-21T01:26:52.642ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
28692024-12-21T01:26:52.642ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:49910 req_id = 76dd5a16-3c41-4c57-9348-a818bf540bdd response_code = 204 uri = /eip-gateways
28702024-12-21T01:26:52.642ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:35958 req_id = 422ca792-22cc-4514-9226-a33cfd1799e4 response_code = 200 uri = /network-bootstore-config
28712024-12-21T01:26:52.652ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 368 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:49910 req_id = 0c8cb0eb-bf49-4efe-a46e-ae1a7e9df5a4 response_code = 204 uri = /vpc-routes
28722024-12-21T01:27:07.080ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:34138
28732024-12-21T01:27:07.081ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 121 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:34138 req_id = f400fad3-1674-4344-b81e-7fb11105e862 response_code = 200 uri = /baseboard
28742024-12-21T01:27:13.392ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8338 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = 81aed843-2dbc-4d77-a1f3-149e5721dd3f response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
28752024-12-21T01:27:21.399ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62191
28762024-12-21T01:27:21.399ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 74 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62191 req_id = dadf2a25-3ad7-486f-9e5c-888f0a7c2648 response_code = 200 uri = /vpc-routes
28772024-12-21T01:27:21.415ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
28782024-12-21T01:27:21.415ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:62191 req_id = 4f691dc8-db01-461e-81e9-219f9f0a47f0 response_code = 204 uri = /eip-gateways
28792024-12-21T01:27:21.486ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62632
28802024-12-21T01:27:21.487ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62632 req_id = b37cea45-cf27-44f4-81a9-548a53e92e5a response_code = 200 uri = /v2p
28812024-12-21T01:27:21.490ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:62191 req_id = e3cb779b-ae7c-4370-ace4-3ffe81338d8a response_code = 204 uri = /vpc-routes
28822024-12-21T01:27:21.578ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47133
28832024-12-21T01:27:21.578ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
28842024-12-21T01:27:21.617ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
28852024-12-21T01:27:21.672ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94542 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:47133 req_id = db66721a-930d-43f3-82e3-ba448b8c6654 response_code = 204 uri = /switch-ports
28862024-12-21T01:27:21.673ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47133 req_id = fa7e407e-c240-4898-8552-19188e9bb700 response_code = 200 uri = /network-bootstore-config
28872024-12-21T01:27:22.547ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:65408
28882024-12-21T01:27:22.548ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 44 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:65408 req_id = c8f139af-fc80-4ea8-bd56-66d33f05ea0f response_code = 200 uri = /vpc-routes
28892024-12-21T01:27:22.550ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47891
28902024-12-21T01:27:22.550ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 20 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47891 req_id = 48f3f38d-5f19-4c88-8ac7-9ce43750fbd4 response_code = 200 uri = /vpc-routes
28912024-12-21T01:27:22.561ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56517
28922024-12-21T01:27:22.561ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 195 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56517 req_id = 1c4ef2ab-3b45-4e51-b93d-156aa8841beb response_code = 200 uri = /v2p
28932024-12-21T01:27:22.593ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
28942024-12-21T01:27:22.593ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47891 req_id = 61530c2e-149e-4465-811f-e4d7b5e3a056 response_code = 204 uri = /eip-gateways
28952024-12-21T01:27:22.596ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
28962024-12-21T01:27:22.596ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 31 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:65408 req_id = 0434070d-ffab-4ab7-8e8a-fe18bd21717f response_code = 204 uri = /eip-gateways
28972024-12-21T01:27:22.602ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47891 req_id = 262ebb8d-8465-4bbd-8993-723ac69959b1 response_code = 204 uri = /vpc-routes
28982024-12-21T01:27:22.606ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:65408 req_id = 33653e8c-e6ec-4dc1-97fc-85fbad554450 response_code = 204 uri = /vpc-routes
28992024-12-21T01:27:22.642ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61088
29002024-12-21T01:27:22.642ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
29012024-12-21T01:27:22.682ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
29022024-12-21T01:27:22.735ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93327 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:61088 req_id = 1bbeb4d6-cf72-4c5c-a201-71efa2fe8f52 response_code = 204 uri = /switch-ports
29032024-12-21T01:27:22.736ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52128
29042024-12-21T01:27:22.736ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 221 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52128 req_id = 218e88b3-d6fb-4722-ae20-d66d59e11853 response_code = 200 uri = /v2p
29052024-12-21T01:27:22.736ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 267 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61088 req_id = 92c3a8d8-1a60-4332-a986-87fe18c034b0 response_code = 200 uri = /network-bootstore-config
29062024-12-21T01:27:22.842ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43212
29072024-12-21T01:27:22.842ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
29082024-12-21T01:27:22.881ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
29092024-12-21T01:27:22.940ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97451 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:43212 req_id = ffa55454-2412-4c30-8815-2da8aeeb5194 response_code = 204 uri = /switch-ports
29102024-12-21T01:27:22.940ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43212 req_id = fa391587-293e-4c53-b1c0-47a213a8d72a response_code = 200 uri = /network-bootstore-config
29112024-12-21T01:27:37.117ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:65524
29122024-12-21T01:27:37.117ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:65524 req_id = c9756fb3-3428-4432-9a78-06f666c24189 response_code = 200 uri = /baseboard
29132024-12-21T01:27:43.392ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7961 local_addr = [fd00:1122:3344:101::1]:42313 method = GET remote_addr = [fd00:1122:3344:101::d]:61296 req_id = 9bba7b06-5c6c-4c3c-aff4-4fee5e8a0f70 response_code = 200 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 uri = /b1eebc13-191d-4a5d-a35e-84ae64fecac9
29142024-12-21T01:27:46.201ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41663
29152024-12-21T01:27:46.202ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41663 req_id = 1375dcf7-6966-469e-a635-946d2340bfc6 response_code = 200 uri = /v2p
29162024-12-21T01:27:46.203ZINFOSledAgent (PortManager): Mapping virtual NIC to physical host file = illumos-utils/src/opte/port_manager.rs:874 mapping = VirtualNetworkInterfaceHost { virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), physical_host_ip: fd00:1122:3344:101::1, vni: Vni(10154056) }
29172024-12-21T01:27:46.203ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1378 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:41663 req_id = 83069c2b-9e98-4264-850a-43e3cb607748 response_code = 204 uri = /v2p
29182024-12-21T01:27:46.288ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35793
29192024-12-21T01:27:46.297ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:555 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 230fd846-fc82-4611-b8a1-b89c61e310fc, kind: Instance { id: 95ae81e6-d851-46fd-bcac-7f515000bd9f }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10154056), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 230fd846-fc82-4611-b8a1-b89c61e310fc, kind: Instance { id: 95ae81e6-d851-46fd-bcac-7f515000bd9f }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10154056), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 230fd846-fc82-4611-b8a1-b89c61e310fc, kind: Instance { id: 95ae81e6-d851-46fd-bcac-7f515000bd9f }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10154056), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(10154056))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 230fd846-fc82-4611-b8a1-b89c61e310fc, kind: Instance { id: 95ae81e6-d851-46fd-bcac-7f515000bd9f }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10154056), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-d4e2afbe9ee", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 62c41e8d-f49b-4cdc-9774-7d850d0e7258, read_only_parent: Some(Volume { block_size: 512, id: 186589e0-2a0a-4693-8983-34b2b086174b, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: c9f7c76a-5a10-49db-bb0f-30ca32b858ca, key: Some("UV3OAu9LOgEU1X2+j2BWlETqlwYK9O5MEkyJNrolqcc="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19002", "[fd00:1122:3344:101::13]:19002", "[fd00:1122:3344:101::15]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 62c41e8d-f49b-4cdc-9774-7d850d0e7258, key: Some("uql8OYVlSJTkppoHfPokWqUfH8uOunxX66xjZVJ2lUw="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19003", "[fd00:1122:3344:101::13]:19003", "[fd00:1122:3344:101::17]:19000"] } }] } }], boot_settings: Some(BootSettings { order: [BootOrderEntry { name: "disk-d4e2afbe9ee" }] }), cloud_init_bytes: Some(..) } instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f metadata = InstanceMetadata { silo_id: 0c5ddff5-9022-4861-af17-d53a460c2ce6, project_id: e2ebdad1-8f04-4214-bdf5-addacd684a7a } migration_id = None propolis_addr = [fd00:1122:3344:101::1:0]:12400 propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0 vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-12-21T01:27:46.288146076Z }
29202024-12-21T01:27:46.297ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:588 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f migration_id = None propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
29212024-12-21T01:27:46.297ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:1281 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f migration_id = None propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0 state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: 230fd846-fc82-4611-b8a1-b89c61e310fc, kind: Instance { id: 95ae81e6-d851-46fd-bcac-7f515000bd9f }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10154056), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 230fd846-fc82-4611-b8a1-b89c61e310fc, kind: Instance { id: 95ae81e6-d851-46fd-bcac-7f515000bd9f }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10154056), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 230fd846-fc82-4611-b8a1-b89c61e310fc, kind: Instance { id: 95ae81e6-d851-46fd-bcac-7f515000bd9f }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10154056), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(10154056))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 230fd846-fc82-4611-b8a1-b89c61e310fc, kind: Instance { id: 95ae81e6-d851-46fd-bcac-7f515000bd9f }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(10154056), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-d4e2afbe9ee", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 62c41e8d-f49b-4cdc-9774-7d850d0e7258, read_only_parent: Some(Volume { block_size: 512, id: 186589e0-2a0a-4693-8983-34b2b086174b, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: c9f7c76a-5a10-49db-bb0f-30ca32b858ca, key: Some("UV3OAu9LOgEU1X2+j2BWlETqlwYK9O5MEkyJNrolqcc="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19002", "[fd00:1122:3344:101::13]:19002", "[fd00:1122:3344:101::15]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 62c41e8d-f49b-4cdc-9774-7d850d0e7258, key: Some("uql8OYVlSJTkppoHfPokWqUfH8uOunxX66xjZVJ2lUw="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::12]:19003", "[fd00:1122:3344:101::13]:19003", "[fd00:1122:3344:101::17]:19000"] } }] } }], boot_settings: Some(BootSettings { order: [BootOrderEntry { name: "disk-d4e2afbe9ee" }] }), cloud_init_bytes: Some(..) }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-12-21T01:27:46.288146076Z }, propolis_addr: [fd00:1122:3344:101::1:0]:12400, migration_id: None }
29222024-12-21T01:27:46.300ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 11869 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35793 req_id = eda2726d-a81a-4043-8ce0-6acb82b07976 response_code = 200 uri = /vmms/0788bf40-d8f1-4726-b93b-237d6e5335e0
29232024-12-21T01:27:46.408ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62642
29242024-12-21T01:27:46.410ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1895 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:62642 req_id = 5d2204c5-500b-4223-b6c7-575ebb291831 response_code = 200 uri = /vmms/0788bf40-d8f1-4726-b93b-237d6e5335e0/state
29252024-12-21T01:27:46.411ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte6", ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 240, 0, 0]), slot: 0, vni: Vni { inner: 10154056 }, subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 } }) }
29262024-12-21T01:27:46.411ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
29272024-12-21T01:27:46.451ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0 file = illumos-utils/src/zone.rs:325 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
29282024-12-21T01:27:46.477ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0 file = illumos-utils/src/zone.rs:363 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
29292024-12-21T01:27:46.574ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34796
29302024-12-21T01:27:46.574ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34796 req_id = 67b82497-360d-44f0-be7e-c9ca06da2532 response_code = 200 uri = /vpc-routes
29312024-12-21T01:27:46.592ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {230fd846-fc82-4611-b8a1-b89c61e310fc: {10.151.2.175: {f87e0ae1-60cd-4de4-8bfc-47510f7fa83c}, 10.151.2.176: {f87e0ae1-60cd-4de4-8bfc-47510f7fa83c}}} }
29322024-12-21T01:27:47.878ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlInstance0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1:0'/> <propval type="astring" name="listen_port" value='12400'/> <propval type="astring" name="metric_addr" value='dns'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
29332024-12-21T01:27:47.878ZINFOSledAgent (InstanceManager): Booting oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0 zone file = illumos-utils/src/running_zone.rs:488 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0 zone = oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0
29342024-12-21T01:27:48.150ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0") failed: Property not found. retry in 67.276743ms file = illumos-utils/src/svc.rs:36 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0 zone = oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0
29352024-12-21T01:27:48.229ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0") failed: Property not found. retry in 124.428975ms file = illumos-utils/src/svc.rs:36 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0 zone = oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0
29362024-12-21T01:27:48.384ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0") failed: Property not found. retry in 172.749861ms file = illumos-utils/src/svc.rs:36 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0 zone = oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0
29372024-12-21T01:27:48.568ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0") failed: Property not found. retry in 211.709615ms file = illumos-utils/src/svc.rs:36 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0 zone = oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0
29382024-12-21T01:27:48.792ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0") failed: Property not found. retry in 723.928517ms file = illumos-utils/src/svc.rs:36 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0 zone = oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0
29392024-12-21T01:27:49.527ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0") failed: Property not found. retry in 689.723629ms file = illumos-utils/src/svc.rs:36 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0 zone = oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0
29402024-12-21T01:27:50.228ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0") failed: Property not found. retry in 664.329771ms file = illumos-utils/src/svc.rs:36 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0 zone = oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0
29412024-12-21T01:27:50.904ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0") failed: Property not found. retry in 1.234521999s file = illumos-utils/src/svc.rs:36 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0 zone = oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0
29422024-12-21T01:27:51.224ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42049
29432024-12-21T01:27:51.314ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33584
29442024-12-21T01:27:51.314ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33584 req_id = 442339a7-5137-4142-9974-1cf202eff8ba response_code = 200 uri = /v2p
29452024-12-21T01:27:51.389ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60737
29462024-12-21T01:27:51.389ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
29472024-12-21T01:27:51.427ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
29482024-12-21T01:27:51.484ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95297 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:60737 req_id = aa4220fc-9629-4deb-83fa-e331c6534703 response_code = 204 uri = /switch-ports
29492024-12-21T01:27:51.485ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60737 req_id = 6b5f6291-2e47-45df-b65f-ec91e06e5d6c response_code = 200 uri = /network-bootstore-config
29502024-12-21T01:27:52.156ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0") failed: Property not found. retry in 1.044379478s file = illumos-utils/src/svc.rs:36 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0 zone = oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0
29512024-12-21T01:27:52.296ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35790
29522024-12-21T01:27:52.370ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33318
29532024-12-21T01:27:52.370ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33318 req_id = 30720dc2-c356-477d-8aa8-b7bc58e5994a response_code = 200 uri = /vpc-routes
29542024-12-21T01:27:52.377ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46784
29552024-12-21T01:27:52.418ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {230fd846-fc82-4611-b8a1-b89c61e310fc: {10.151.2.176: {f87e0ae1-60cd-4de4-8bfc-47510f7fa83c}, 10.151.2.175: {f87e0ae1-60cd-4de4-8bfc-47510f7fa83c}}} }
29562024-12-21T01:27:52.418ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:33318 req_id = 2ebc57aa-dbff-4f6d-83ff-4eca0917296e response_code = 204 uri = /eip-gateways
29572024-12-21T01:27:52.462ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62639
29582024-12-21T01:27:52.463ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62639 req_id = 981ffa23-6f5f-4ee0-8e53-a2372dc2a527 response_code = 200 uri = /vpc-routes
29592024-12-21T01:27:52.463ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48217
29602024-12-21T01:27:52.463ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
29612024-12-21T01:27:52.508ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
29622024-12-21T01:27:52.570ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106809 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:48217 req_id = 0b487d8d-d5bc-4384-a1aa-328526eec52a response_code = 204 uri = /switch-ports
29632024-12-21T01:27:52.570ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49477
29642024-12-21T01:27:52.570ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48257
29652024-12-21T01:27:52.570ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 215 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48217 req_id = 4df9d48e-2978-453c-a308-0343924aec07 response_code = 200 uri = /network-bootstore-config
29662024-12-21T01:27:52.571ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64312
29672024-12-21T01:27:52.571ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {230fd846-fc82-4611-b8a1-b89c61e310fc: {10.151.2.176: {f87e0ae1-60cd-4de4-8bfc-47510f7fa83c}, 10.151.2.175: {f87e0ae1-60cd-4de4-8bfc-47510f7fa83c}}} }
29682024-12-21T01:27:52.571ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 620 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:62639 req_id = ff462bf4-0a92-4d4c-b659-48ce73e8caa6 response_code = 204 uri = /eip-gateways
29692024-12-21T01:27:52.571ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4134
29702024-12-21T01:27:52.571ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 717 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49477 req_id = a71e532e-946d-45e7-af54-b1a9b2080c33 response_code = 200 uri = /v2p
29712024-12-21T01:27:52.571ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 223 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64312 req_id = 1417f6fe-776f-4850-9437-9bcb5b7cbcd6 response_code = 200 uri = /v2p
29722024-12-21T01:27:52.605ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5012 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:33318 req_id = 98ad6522-1407-47dc-b9f3-5a97e5e0514a response_code = 204 uri = /vpc-routes
29732024-12-21T01:27:52.614ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4158
29742024-12-21T01:27:52.672ZINFOSledAgent (PortManager): skipping delta compute for subnet file = illumos-utils/src/opte/port_manager.rs:486 new_vers = RouterVersion { router_id: 4544fc72-c524-437a-804e-dd7d250a26c0, version: 3 } old_vers = RouterVersion { router_id: 4544fc72-c524-437a-804e-dd7d250a26c0, version: 3 } subnet = RouterId { vni: Vni(10154056), kind: System }
29752024-12-21T01:27:52.672ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1043 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:62639 req_id = 27e5cf8c-36e2-4d5b-a050-84743c4e39a0 response_code = 204 uri = /vpc-routes
29762024-12-21T01:27:52.681ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 109884 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:48257 req_id = 9f724ba9-8e02-4dba-a136-bcd862a7989f response_code = 204 uri = /switch-ports
29772024-12-21T01:27:52.681ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48257 req_id = dd092d52-761c-4ea3-98e0-d6df4401a67a response_code = 200 uri = /network-bootstore-config
29782024-12-21T01:27:53.236ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0 file = sled-agent/src/instance.rs:1786 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
29792024-12-21T01:27:53.248ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0") failed: Property not found. retry in 56.921686ms file = illumos-utils/src/svc.rs:36 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
29802024-12-21T01:27:53.315ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0") failed: Property not found. retry in 144.986104ms file = illumos-utils/src/svc.rs:36 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
29812024-12-21T01:27:53.472ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0") failed: Property not found. retry in 278.063646ms file = illumos-utils/src/svc.rs:36 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
29822024-12-21T01:27:53.761ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:1795 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
29832024-12-21T01:27:53.799ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:1819 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
29842024-12-21T01:27:53.938ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:932 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
29852024-12-21T01:27:53.939ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1249 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
29862024-12-21T01:27:53.939ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1561366 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46784 req_id = bace88fe-b61c-4eae-a655-f718bdfaae87 response_code = 200 uri = /vmms/0788bf40-d8f1-4726-b93b-237d6e5335e0/state
29872024-12-21T01:27:53.939ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7346565 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34796 req_id = f8170e08-2589-4754-b104-5b8704288f71 response_code = 204 uri = /eip-gateways
29882024-12-21T01:27:53.939ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2714796 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42049 req_id = e796f104-d6df-4049-9c3d-b8af4aac9220 response_code = 200 uri = /vmms/0788bf40-d8f1-4726-b93b-237d6e5335e0/state
29892024-12-21T01:27:53.939ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1642700 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35790 req_id = b3155279-21b5-4856-baa0-0b4f9adaa56c response_code = 200 uri = /vmms/0788bf40-d8f1-4726-b93b-237d6e5335e0/state
29902024-12-21T01:27:53.939ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Creating), migration_in: None, migration_out: None, time: 2024-12-21T01:27:53.939398473Z } file = sled-agent/src/instance.rs:780 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
29912024-12-21T01:27:53.939ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-12-21T01:27:53.939398473Z } propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
29922024-12-21T01:27:53.940ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0 state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-12-21T01:27:53.939398473Z }, migration_in: None, migration_out: None }
29932024-12-21T01:27:53.983ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_in: None, migration_out: None, time: 2024-12-21T01:27:53.983567757Z } file = sled-agent/src/instance.rs:780 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
29942024-12-21T01:27:53.983ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-12-21T01:27:53.983567757Z } propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
29952024-12-21T01:27:53.983ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0 state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-12-21T01:27:53.983567757Z }, migration_in: None, migration_out: None }
29962024-12-21T01:27:54.055ZINFOSledAgent (PortManager): skipping delta compute for subnet file = illumos-utils/src/opte/port_manager.rs:486 new_vers = RouterVersion { router_id: 4544fc72-c524-437a-804e-dd7d250a26c0, version: 3 } old_vers = RouterVersion { router_id: 4544fc72-c524-437a-804e-dd7d250a26c0, version: 3 } subnet = RouterId { vni: Vni(10154056), kind: System }
29972024-12-21T01:27:54.055ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 218 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:34796 req_id = 37ec9102-b884-455d-9903-ff57914586cc response_code = 204 uri = /vpc-routes
29982024-12-21T01:27:54.096ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49188
29992024-12-21T01:27:54.096ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 31 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49188 req_id = b7bbf940-de58-43eb-9e2d-c0a1ca22eae1 response_code = 200 uri = /vpc-routes
30002024-12-21T01:27:54.111ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {230fd846-fc82-4611-b8a1-b89c61e310fc: {10.151.2.176: {f87e0ae1-60cd-4de4-8bfc-47510f7fa83c}, 10.151.2.175: {f87e0ae1-60cd-4de4-8bfc-47510f7fa83c}}} }
30012024-12-21T01:27:54.111ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 52 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49188 req_id = 2d127ef2-9cfd-4688-8f22-091403eed589 response_code = 204 uri = /eip-gateways
30022024-12-21T01:27:54.125ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49188 req_id = a8b5737c-e664-4769-86ce-7c140204094d response_code = 204 uri = /vpc-routes
30032024-12-21T01:28:04.085ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62506
30042024-12-21T01:28:04.086ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 506 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:62506 req_id = 755a5168-fc6c-467c-81e3-22e8f39f9237 response_code = 200 uri = /vmms/0788bf40-d8f1-4726-b93b-237d6e5335e0/state
30052024-12-21T01:28:04.086ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_in: None, migration_out: None, time: 2024-12-21T01:28:04.086134302Z } file = sled-agent/src/instance.rs:780 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
30062024-12-21T01:28:04.086ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-12-21T01:28:04.086134302Z } propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
30072024-12-21T01:28:04.086ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0 state = SledVmmState { vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-12-21T01:28:04.086134302Z }, migration_in: None, migration_out: None }
30082024-12-21T01:28:04.155ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_in: None, migration_out: None, time: 2024-12-21T01:28:04.155860956Z } file = sled-agent/src/instance.rs:780 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
30092024-12-21T01:28:04.155ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-12-21T01:28:04.155860956Z } propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
30102024-12-21T01:28:04.155ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:816 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
30112024-12-21T01:28:04.156ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
30122024-12-21T01:28:04.156ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:602 id = 17310151093260669461 n_samples = 6 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
30132024-12-21T01:28:04.156ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:602 id = 15065535597835302039 n_samples = 6 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
30142024-12-21T01:28:04.156ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:607
30152024-12-21T01:28:04.156ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/1bf7a526-12df-41e0-8b64-89b419ddc8ee/debug/bundle/zone", "/pool/int/39f3a11f-ebbe-4984-ad03-60e64a112dfb/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/3d4d0721-dd16-40d1-afff-d64c8f3a10ff/crypt/debug", "/pool/ext/41cc83ed-adbe-492f-ae31-7074b31692b0/crypt/debug", "/pool/ext/7e02d0c4-4e5d-440d-94c1-593a67af3118/crypt/debug", "/pool/ext/22c20cb3-3659-48b7-b12d-b9ae21224e50/crypt/debug", "/pool/ext/10325d85-ce1b-49aa-8039-7e9482ec8803/crypt/debug", "/pool/ext/2d3f9ebb-9443-4b48-b1ee-fcd55a3fd739/crypt/debug", "/pool/ext/192cce10-69a8-4ed6-a459-9ca86a7206ec/crypt/debug", "/pool/ext/34789575-52b6-411b-9614-90f11629cb04/crypt/debug", "/pool/ext/e80b2509-2264-41c1-be0f-a3173a243a47/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:339 zone_name = oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0
30162024-12-21T01:28:04.788ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1142 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0", bundle_id: fff451ec-bef0-4a2c-8b8e-8afe35c4d1f5 }, time_created: 2024-12-21T01:28:04.156840783Z, version: 0, cause: TerminatedInstance }
30172024-12-21T01:28:04.788ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_0788bf40-d8f1-4726-b93b-237d6e5335e0 file = sled-agent/src/instance.rs:1032 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
30182024-12-21T01:28:06.569ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:279 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
30192024-12-21T01:28:06.587ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:1000 id = RouterId { vni: Vni(10154056), kind: System }
30202024-12-21T01:28:06.587ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:1000 id = RouterId { vni: Vni(10154056), kind: Custom(V4(Ipv4Net { addr: 172.30.0.0, width: 22 })) }
30212024-12-21T01:28:06.589ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-12-21T01:28:06.589063592Z }, migration_in: None, migration_out: None }
30222024-12-21T01:28:06.612ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0 state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-12-21T01:28:06.589063592Z }, migration_in: None, migration_out: None }
30232024-12-21T01:28:06.616ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:969 instance_id = 95ae81e6-d851-46fd-bcac-7f515000bd9f propolis_id = 0788bf40-d8f1-4726-b93b-237d6e5335e0
30242024-12-21T01:28:07.112ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62952
30252024-12-21T01:28:07.113ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62952 req_id = fc9828f7-d27f-4749-86a4-c999175dc322 response_code = 200 uri = /vpc-routes
30262024-12-21T01:28:07.154ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:63840
30272024-12-21T01:28:07.154ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:63840 req_id = f798bbe3-abb2-4afa-93a7-0b7880b2b123 response_code = 200 uri = /baseboard
30282024-12-21T01:28:07.175ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
30292024-12-21T01:28:07.175ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1249 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9
30302024-12-21T01:28:07.175ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:62952 req_id = e0f1fbc0-161e-40f8-99f8-e988bd7016a4 response_code = 204 uri = /eip-gateways
30312024-12-21T01:28:07.186ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2047 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:62952 req_id = e8d40dac-7316-48eb-a1a8-4b3031f4c82a response_code = 204 uri = /vpc-routes
30322024-12-21T01:28:07.326ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:52024
30332024-12-21T01:28:07.328ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:52024 req_id = dd46dd54-157c-46f2-bde4-41cf41d98e5c response_code = 200 uri = /v2p
30342024-12-21T01:28:07.409ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46138
30352024-12-21T01:28:07.409ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46138 req_id = bda3789e-47d8-4f07-b2da-ddfc2622dd80 response_code = 200 uri = /v2p
30362024-12-21T01:28:07.449ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38661
30372024-12-21T01:28:07.449ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 32 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38661 req_id = e832b84d-e471-4e98-aa2a-b11c49d41890 response_code = 200 uri = /vpc-routes
30382024-12-21T01:28:07.464ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
30392024-12-21T01:28:07.464ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 48 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38661 req_id = c3e98ff3-8e45-4e27-8dba-4b0192958f3e response_code = 204 uri = /eip-gateways
30402024-12-21T01:28:07.472ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:38661 req_id = c7b569e0-4e63-4a20-a3ea-d6e8e79ba58f response_code = 204 uri = /vpc-routes
30412024-12-21T01:28:08.123ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60154
30422024-12-21T01:28:08.123ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60154 req_id = c2cac2a3-5569-4764-a8a3-9610c8754407 response_code = 200 uri = /vpc-routes
30432024-12-21T01:28:08.136ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1223 sled_id = b1eebc13-191d-4a5d-a35e-84ae64fecac9 values = ExternalIpGatewayMap { mappings: {} }
30442024-12-21T01:28:08.136ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:60154 req_id = 72184374-fe8c-4e91-bcba-198d541a280a response_code = 204 uri = /eip-gateways
30452024-12-21T01:28:08.144ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 130 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:60154 req_id = 0d19c737-cfdd-46c7-bc7f-24b345f6413a response_code = 204 uri = /vpc-routes