Max level shown:
1 [ Dec 13 22:27:07 Enabled. ]
2 [ Dec 13 22:27:07 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/sled-agent/sled-agent run /opt/oxide/sled-agent/pkg/config.toml &"). ]
3 [ Dec 13 22:27:07 Method "start" exited with status 0. ]
4 [ Dec 13 22:27:07 Rereading configuration. ]
5 [ Dec 13 22:27:07 No 'refresh' method defined. Treating as :true. ]
6 note: configured to log to "/dev/stdout"
72024-12-13T22:27:07.472ZINFOSledAgent: Ensuring contract reaper thread file = illumos-utils/src/running_zone.rs:113
82024-12-13T22:27:07.472ZINFOSledAgent (BootstrapAgentStartup): finding links ["net0", "net1"] file = sled-agent/src/bootstrap/pre_server.rs:255
92024-12-13T22:27:07.526ZINFOSledAgent (BootstrapAgentStartup): Starting mg-ddm service addr-objs = [AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }] file = sled-agent/src/bootstrap/pre_server.rs:262
102024-12-13T22:27:07.526ZINFOSledAgent (BootstrapAgentStartup): Importing mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:42 path = /opt/oxide/mg-ddm/pkg/ddm/manifest.xml
112024-12-13T22:27:07.640ZINFOSledAgent (BootstrapAgentStartup): Setting mg-ddm interfaces file = sled-agent/src/bootstrap/maghemite.rs:50 interfaces = ("net0/ll" "net1/ll")
122024-12-13T22:27:07.649ZINFOSledAgent (BootstrapAgentStartup): Enabling mg-ddm service file = sled-agent/src/bootstrap/maghemite.rs:56
132024-12-13T22:27:07.657ZINFOSledAgent (BootstrapAgentStartup): Ensuring zfs key directory exists file = sled-agent/src/bootstrap/pre_server.rs:272 path = /var/run/oxide/
142024-12-13T22:27:08.261ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fdb0:18c0:4d0d:a02a::, width: 64 }
152024-12-13T22:27:08.404ZINFOSledAgent: Starting KeyManager file = sled-agent/src/long_running_tasks.rs:126
162024-12-13T22:27:08.404ZINFOSledAgent: Starting StorageManager file = sled-agent/src/long_running_tasks.rs:138
172024-12-13T22:27:08.404ZINFOSledAgent: Starting StorageMonitor file = sled-agent/src/long_running_tasks.rs:151
182024-12-13T22:27:08.404ZINFOSledAgent (DumpSetup-worker): DumpSetup poll loop started. file = sled-agent/src/dump_setup.rs:520
192024-12-13T22:27:08.404ZINFOSledAgent: Starting HardwareManager file = sled-agent/src/long_running_tasks.rs:173 nongimlet_observed_disks = [] sled_mode = Scrimlet { asic: SoftNpuZone }
202024-12-13T22:27:08.404ZINFOSledAgent (HardwareManager): Creating HardwareManager file = sled-hardware/src/illumos/mod.rs:717
212024-12-13T22:27:08.407ZINFOSledAgent (HardwareManager): Generated i86pc baseboard Pc { identifier: "sock", model: "i86pc" } file = sled-hardware/src/illumos/mod.rs:624
222024-12-13T22:27:08.407ZWARNSledAgent (HardwareManager): Device is not a Gimlet (i86pc), proceeding with null hardware view file = sled-hardware/src/illumos/mod.rs:766
232024-12-13T22:27:08.407ZINFOSledAgent: Starting HardwareMonitor file = sled-agent/src/long_running_tasks.rs:187
242024-12-13T22:27:08.407ZINFOSledAgent (HardwareManager): Monitoring for hardware updates file = sled-hardware/src/illumos/mod.rs:825
252024-12-13T22:27:08.407ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_0.vdev
262024-12-13T22:27:08.407ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = None
272024-12-13T22:27:08.408ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
282024-12-13T22:27:08.408ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
292024-12-13T22:27:08.408ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } path = /scratch/m2_0.vdev
302024-12-13T22:27:08.420ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_0.vdev file = sled-hardware/src/disk.rs:367
312024-12-13T22:27:08.420ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = 6c407cb8-0013-4482-9b28-b5bd52e5b10f (zpool)
322024-12-13T22:27:08.523ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 6c407cb8-0013-4482-9b28-b5bd52e5b10f (zpool), kind: Internal }
332024-12-13T22:27:08.997ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:603
342024-12-13T22:27:08.997ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/m2_1.vdev
352024-12-13T22:27:08.997ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
362024-12-13T22:27:08.997ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
372024-12-13T22:27:08.997ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
382024-12-13T22:27:08.997ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
392024-12-13T22:27:08.997ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } path = /scratch/m2_1.vdev
402024-12-13T22:27:09.006ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/m2_1.vdev file = sled-hardware/src/disk.rs:367
412024-12-13T22:27:09.006ZINFOSledAgent (StorageResources): Formatting zpool with generated ID file = sled-hardware/src/disk.rs:378 id = dbb4b417-a6b8-4743-be7a-9642ba91ec5c (zpool)
422024-12-13T22:27:09.098ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: dbb4b417-a6b8-4743-be7a-9642ba91ec5c (zpool), kind: Internal }
432024-12-13T22:27:09.568ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_0.vdev
442024-12-13T22:27:09.568ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6c407cb8-0013-4482-9b28-b5bd52e5b10f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: dbb4b417-a6b8-4743-be7a-9642ba91ec5c (zpool), kind: Internal } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
452024-12-13T22:27:09.569ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
462024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
472024-12-13T22:27:09.569ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
482024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
492024-12-13T22:27:09.569ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_1.vdev
502024-12-13T22:27:09.569ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
512024-12-13T22:27:09.569ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
522024-12-13T22:27:09.569ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6c407cb8-0013-4482-9b28-b5bd52e5b10f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: dbb4b417-a6b8-4743-be7a-9642ba91ec5c (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
532024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
542024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
552024-12-13T22:27:09.569ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_2.vdev
562024-12-13T22:27:09.569ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
572024-12-13T22:27:09.569ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6c407cb8-0013-4482-9b28-b5bd52e5b10f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: dbb4b417-a6b8-4743-be7a-9642ba91ec5c (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
582024-12-13T22:27:09.569ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
592024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
602024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
612024-12-13T22:27:09.569ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_3.vdev
622024-12-13T22:27:09.569ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6c407cb8-0013-4482-9b28-b5bd52e5b10f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: dbb4b417-a6b8-4743-be7a-9642ba91ec5c (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
632024-12-13T22:27:09.569ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
642024-12-13T22:27:09.569ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
652024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
662024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
672024-12-13T22:27:09.569ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_4.vdev
682024-12-13T22:27:09.569ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6c407cb8-0013-4482-9b28-b5bd52e5b10f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: dbb4b417-a6b8-4743-be7a-9642ba91ec5c (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
692024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
702024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
712024-12-13T22:27:09.569ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
722024-12-13T22:27:09.569ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
732024-12-13T22:27:09.569ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_5.vdev
742024-12-13T22:27:09.569ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6c407cb8-0013-4482-9b28-b5bd52e5b10f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: dbb4b417-a6b8-4743-be7a-9642ba91ec5c (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
752024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
762024-12-13T22:27:09.569ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
772024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
782024-12-13T22:27:09.569ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
792024-12-13T22:27:09.569ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_6.vdev
802024-12-13T22:27:09.569ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6c407cb8-0013-4482-9b28-b5bd52e5b10f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: dbb4b417-a6b8-4743-be7a-9642ba91ec5c (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
812024-12-13T22:27:09.569ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
822024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
832024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
842024-12-13T22:27:09.569ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
852024-12-13T22:27:09.569ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_7.vdev
862024-12-13T22:27:09.569ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
872024-12-13T22:27:09.569ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6c407cb8-0013-4482-9b28-b5bd52e5b10f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: dbb4b417-a6b8-4743-be7a-9642ba91ec5c (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
882024-12-13T22:27:09.569ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
892024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
902024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
912024-12-13T22:27:09.569ZINFOSledAgent: Upserting synthetic device to Storage Manager file = sled-agent/src/long_running_tasks.rs:243 vdev = /scratch/u2_8.vdev
922024-12-13T22:27:09.569ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6c407cb8-0013-4482-9b28-b5bd52e5b10f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: dbb4b417-a6b8-4743-be7a-9642ba91ec5c (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
932024-12-13T22:27:09.569ZINFOSledAgent (StorageManager): Received DetectedRawDisk { raw_disk: Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }), tx: ... } file = sled-storage/src/manager.rs:603
942024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
952024-12-13T22:27:09.569ZINFOSledAgent (StorageResources): Inserting disk file = sled-storage/src/resources.rs:457 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
962024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
972024-12-13T22:27:09.569ZINFOSledAgent: Waiting for boot disk file = sled-agent/src/long_running_tasks.rs:97
982024-12-13T22:27:09.569ZINFOSledAgent: Found boot disk DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" } file = sled-agent/src/long_running_tasks.rs:99
992024-12-13T22:27:09.569ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(1), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6c407cb8-0013-4482-9b28-b5bd52e5b10f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: dbb4b417-a6b8-4743-be7a-9642ba91ec5c (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: Unmanaged(Synthetic(RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
1002024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
1012024-12-13T22:27:09.569ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
1022024-12-13T22:27:09.569ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1032024-12-13T22:27:09.569ZINFOSledAgent: Starting Bootstore file = sled-agent/src/long_running_tasks.rs:211
1042024-12-13T22:27:09.569ZINFOSledAgent (bootstore): No ledger in /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1052024-12-13T22:27:09.569ZINFOSledAgent (bootstore): No ledger in /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/cluster/bootstore-fsm-state.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1062024-12-13T22:27:09.569ZINFOSledAgent (bootstore): No ledger found. Loading Fsm::State as Uninitialized file = bootstore/src/schemes/v0/storage.rs:83 peer_id = pc-sock-i86pc
1072024-12-13T22:27:09.569ZINFOSledAgent (bootstore): No ledger in /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1082024-12-13T22:27:09.569ZINFOSledAgent (bootstore): No ledger in /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/cluster/bootstore-network-config.json file = common/src/ledger.rs:188 peer_id = pc-sock-i86pc
1092024-12-13T22:27:09.569ZINFOSledAgent (bootstore): No ledger found for network config file = bootstore/src/schemes/v0/storage.rs:146 peer_id = pc-sock-i86pc
1102024-12-13T22:27:09.569ZINFOSledAgent: Starting Bootstore DDMD poller file = sled-agent/src/long_running_tasks.rs:216
1112024-12-13T22:27:09.569ZINFOSledAgent: Starting ZoneBundler related tasks file = sled-agent/src/long_running_tasks.rs:231
1122024-12-13T22:27:09.580ZINFOSledAgent (ServiceManager): Creating ServiceManager file = sled-agent/src/services.rs:736
1132024-12-13T22:27:09.580ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 1373, tv_nsec: 453695471 } time_to_next_cleanup = 599.999908808s
1142024-12-13T22:27:09.580ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3947
1152024-12-13T22:27:09.581ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1162024-12-13T22:27:09.581ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/sled-agent-request.json file = common/src/ledger.rs:188
1172024-12-13T22:27:09.583ZINFOSledAgent (BootstrapAgentStartup): No ledger in /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/sled-agent-request.json file = common/src/ledger.rs:188
1182024-12-13T22:27:09.584ZINFOSledAgent (dropshot (BootstrapAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fdb0:18c0:4d0d:a02a::1]:80
1192024-12-13T22:27:09.586ZINFOSledAgent (SledAgentSprocketsServer): Started listening file = sled-agent/src/bootstrap/sprockets_server.rs:51 local_addr = [fdb0:18c0:4d0d:a02a::1]:12346
1202024-12-13T22:27:09.586ZINFOSledAgent (RSS): Injecting RSS configuration: RackInitializeRequest { trust_quorum_peers: None, bootstrap_discovery: OnlyOurs, ntp_servers: [ "0.pool.ntp.org", ], dns_servers: [ 1.1.1.1, 9.9.9.9, ], internal_services_ip_pool_ranges: [ V4( Ipv4Range { first: 10.151.2.164, last: 10.151.2.173, }, ), ], external_dns_ips: [ 10.151.2.164, 10.151.2.165, ], external_dns_zone_name: "oxide.test", external_certificates: "<redacted>", recovery_silo: RecoverySiloConfig { silo_name: Name( "recovery", ), user_name: UserId( "recovery", ), user_password_hash: NewPasswordHash( PasswordHashString { string: "$argon2id$v=19$m=98304,t=13,p=1$RUlWc0ZxaHo0WFdrN0N6ZQ$S8p52j85GPvMhR/ek3GL0el/oProgTwWpHJZ8lsQQoY", encoding: B64, }, ), }, rack_network_config: RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, vlan_id: None, rib_priority: None, }, ], addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: Some( None, ), bgp_peers: [], autoneg: false, lldp: None, tx_eq: None, }, ], bgp: [], bfd: [], }, allowed_source_ips: Any, } file = sled-agent/src/rack_setup/service.rs:1203
1212024-12-13T22:27:09.586ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1222024-12-13T22:27:09.586ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1232024-12-13T22:27:09.586ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1242024-12-13T22:27:09.586ZINFOSledAgent (RSS): No ledger in /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/rss-started.marker file = common/src/ledger.rs:188
1252024-12-13T22:27:09.586ZINFOSledAgent (RSS): No ledger in /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/rss-started.marker file = common/src/ledger.rs:188
1262024-12-13T22:27:09.586ZINFOSledAgent (RSS): No ledger in /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1272024-12-13T22:27:09.586ZINFOSledAgent (RSS): No ledger in /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/rss-plan-completed.marker file = common/src/ledger.rs:188
1282024-12-13T22:27:09.586ZINFOSledAgent (RSS): RSS not previously run. Creating plans. file = sled-agent/src/rack_setup/service.rs:1252
1292024-12-13T22:27:09.586ZINFOSledAgent (RSS): Writing ledger to /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/.rss-started.marker.tmp file = common/src/ledger.rs:199
1302024-12-13T22:27:09.586ZINFOSledAgent (RSS): Writing ledger to /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/.rss-started.marker.tmp file = common/src/ledger.rs:199
1312024-12-13T22:27:09.587ZINFOSledAgent (RSS): Creating new allocation plan file = sled-agent/src/rack_setup/service.rs:1284
1322024-12-13T22:27:09.587ZINFOSledAgent (RSS): Serializing plan file = sled-agent/src/rack_setup/plan/sled.rs:66
1332024-12-13T22:27:09.587ZINFOSledAgent (RSS): Creating plan for the sled at fdb0:18c0:4d0d:a02a::1 file = sled-agent/src/rack_setup/plan/sled.rs:39
1342024-12-13T22:27:09.587ZINFOSledAgent (RSS): Writing Rack Network Configuration to bootstore file = sled-agent/src/rack_setup/service.rs:1316
1352024-12-13T22:27:09.587ZINFOSledAgent (bootstore): Attempting to update network config with generation: 1, current_generation: 0 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
1362024-12-13T22:27:09.587ZINFOSledAgent (bootstore): Writing ledger to /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1372024-12-13T22:27:09.587ZINFOSledAgent (bootstore): Writing ledger to /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
1382024-12-13T22:27:09.587ZINFOSledAgent (bootstore): Broadcasting network config with generation 1 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
1392024-12-13T22:27:09.587ZINFOSledAgent (BootstrapAgentRssHandler): Received initialization request from RSS file = sled-agent/src/bootstrap/rss_handle.rs:232 request = StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 35eb16d3-adec-4806-a15c-ab7a15d18765 (sled), rack_id: 984ee802-6f4e-4d6b-aed6-da91b38cb72b, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
1402024-12-13T22:27:09.588ZINFOSledAgent (ServiceManager): Enabling switch zone (new) file = sled-agent/src/services.rs:4244
1412024-12-13T22:27:09.588ZINFOSledAgent (ServiceManager): Starting switch zone file = sled-agent/src/services.rs:4644
1422024-12-13T22:27:09.592ZINFOSledAgent (SledAgentSprocketsClient): Loaded keys and certs BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/client.rs:50
1432024-12-13T22:27:09.598ZINFOSledAgent (SledAgentSprocketsClient): Certificate chain verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1442024-12-13T22:27:09.598ZINFOSledAgent (SledAgentSprocketsClient): Signature verified successfully BootstrapAgentClient = [fdb0:18c0:4d0d:a02a::1]:12346 file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1452024-12-13T22:27:09.603ZINFOSledAgent (SledAgentSprocketsServer): Certificate chain verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:307
1462024-12-13T22:27:09.603ZINFOSledAgent (SledAgentSprocketsServer): Signature verified successfully file = /home/build/.cargo/git/checkouts/sprockets-82488b5b61bf00db/cc13773/tls/src/keys.rs:376
1472024-12-13T22:27:09.603ZINFOSledAgent (SledAgentSprocketsServer): Accepted connection file = sled-agent/src/bootstrap/sprockets_server.rs:74 remote_addr = [fdb0:18c0:4d0d:a02a::1]:65160
1482024-12-13T22:27:09.603ZINFOSledAgent (SledAgentMain): Loading Sled Agent: StartSledAgentRequest { generation: 0, schema_version: 1, body: StartSledAgentRequestBody { id: 35eb16d3-adec-4806-a15c-ab7a15d18765 (sled), rack_id: 984ee802-6f4e-4d6b-aed6-da91b38cb72b, use_trust_quorum: false, is_lrtq_learner: false, subnet: Ipv6Subnet { net: Ipv6Net { addr: fd00:1122:3344:101::, width: 64 } } } } file = sled-agent/src/bootstrap/server.rs:362
1492024-12-13T22:27:09.603ZINFOSledAgent (SledAgentMain): KeyManager: using hardcoded secret retriever file = sled-agent/src/bootstrap/server.rs:377
1502024-12-13T22:27:09.603ZINFOSledAgent: setting up sled agent server file = sled-agent/src/server.rs:44
1512024-12-13T22:27:09.603ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
1522024-12-13T22:27:09.603ZINFOSledAgent (StorageManager): Received KeyManagerReady file = sled-storage/src/manager.rs:603
1532024-12-13T22:27:09.603ZINFOSledAgent (StorageManager): Loading storage config file = sled-storage/src/manager.rs:809
1542024-12-13T22:27:09.603ZINFOSledAgent (StorageManager): No ledger in /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1552024-12-13T22:27:09.603ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:101::, width: 64 }
1562024-12-13T22:27:09.603ZINFOSledAgent (StorageManager): No ledger in /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = load_disks_ledger
1572024-12-13T22:27:09.603ZINFOSledAgent (StorageManager): No ledger of physical disks exists file = sled-storage/src/manager.rs:798
1582024-12-13T22:27:09.603ZINFOSledAgent (StorageManager): KeyManager ready, but no ledger detected file = sled-storage/src/manager.rs:826
1592024-12-13T22:27:09.603ZINFOSledAgent (StorageManager): automatically managing disks file = sled-storage/src/manager.rs:592
1602024-12-13T22:27:09.603ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
1612024-12-13T22:27:09.603ZINFOSledAgent (StorageManager): Successfully synchronized disks without error file = sled-storage/src/manager.rs:697 result = DisksManagementResult { status: [] }
1622024-12-13T22:27:09.622ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1632024-12-13T22:27:09.644ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:325
1642024-12-13T22:27:09.661ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_switch file = illumos-utils/src/zone.rs:363
1652024-12-13T22:27:09.687ZINFOSledAgent: SledAgent::new(..) starting file = sled-agent/src/sled_agent.rs:412 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
1662024-12-13T22:27:09.687ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
1672024-12-13T22:27:09.687ZINFOSledAgent: Requested swap device of size 64 GiB file = sled-agent/src/sled_agent.rs:424 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
1682024-12-13T22:27:09.687ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1692024-12-13T22:27:09.690ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 142.168833ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "62874d80-776c-4dbb-be3c-0731a1322e49", "content-length": "124", "date": "Fri, 13 Dec 2024 22:27:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "62874d80-776c-4dbb-be3c-0731a1322e49" } file = clients/ddm-admin-client/src/lib.rs:177
1702024-12-13T22:27:09.701ZINFOSledAgent: attempting to create encrypted zvol: name="oxi_6c407cb8-0013-4482-9b28-b5bd52e5b10f/swap", size_gb=64 file = sled-agent/src/swap_device.rs:153
1712024-12-13T22:27:09.725ZINFOSledAgent: successfully created encrypted zvol: name="oxi_6c407cb8-0013-4482-9b28-b5bd52e5b10f/swap", size_gb=64 file = sled-agent/src/swap_device.rs:249
1722024-12-13T22:27:09.725ZINFOSledAgent: adding swap device: swapname="/dev/zvol/dsk/oxi_6c407cb8-0013-4482-9b28-b5bd52e5b10f/swap" file = sled-agent/src/swap_device.rs:96
1732024-12-13T22:27:09.730ZINFOSledAgent: Mounting backing filesystems file = sled-agent/src/sled_agent.rs:439 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
1742024-12-13T22:27:09.730ZINFOSledAgent (BackingFs): Processing fmd file = sled-agent/src/backing_fs.rs:128
1752024-12-13T22:27:09.730ZINFOSledAgent (BackingFs): Ensuring dataset oxi_6c407cb8-0013-4482-9b28-b5bd52e5b10f/backing/fmd file = sled-agent/src/backing_fs.rs:138
1762024-12-13T22:27:09.813ZINFOSledAgent (BackingFs): Stopping service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:173
1772024-12-13T22:27:09.834ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1782024-12-13T22:27:09.834ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 69.575442ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "86788786-445a-4304-ab80-8ac587f150e0", "content-length": "124", "date": "Fri, 13 Dec 2024 22:27:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "86788786-445a-4304-ab80-8ac587f150e0" } file = clients/ddm-admin-client/src/lib.rs:177
1792024-12-13T22:27:09.904ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1802024-12-13T22:27:09.905ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 80.143829ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "010077fa-34b7-4de2-89cb-e0274699b64c", "content-length": "124", "date": "Fri, 13 Dec 2024 22:27:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "010077fa-34b7-4de2-89cb-e0274699b64c" } file = clients/ddm-admin-client/src/lib.rs:177
1812024-12-13T22:27:09.986ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1822024-12-13T22:27:09.986ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 135.888271ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "6a729d38-0b0c-42b5-8f87-ce095d6d547d", "content-length": "124", "date": "Fri, 13 Dec 2024 22:27:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "6a729d38-0b0c-42b5-8f87-ce095d6d547d" } file = clients/ddm-admin-client/src/lib.rs:177
1832024-12-13T22:27:10.031ZINFOSledAgent (BackingFs): Mounting oxi_6c407cb8-0013-4482-9b28-b5bd52e5b10f/backing/fmd on /var/fm/fmd file = sled-agent/src/backing_fs.rs:181
1842024-12-13T22:27:10.045ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/rsrc file = sled-agent/src/backing_fs.rs:189
1852024-12-13T22:27:10.045ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/ckpt file = sled-agent/src/backing_fs.rs:189
1862024-12-13T22:27:10.045ZINFOSledAgent (BackingFs): Ensuring directory /var/fm/fmd/xprt file = sled-agent/src/backing_fs.rs:189
1872024-12-13T22:27:10.045ZINFOSledAgent (BackingFs): Starting service svc:/system/fmd:default file = sled-agent/src/backing_fs.rs:195
1882024-12-13T22:27:10.124ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1892024-12-13T22:27:10.124ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 248.204758ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "b28e45c9-00d3-4884-9523-5b508937d099", "content-length": "124", "date": "Fri, 13 Dec 2024 22:27:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "b28e45c9-00d3-4884-9523-5b508937d099" } file = clients/ddm-admin-client/src/lib.rs:177
1902024-12-13T22:27:10.374ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1912024-12-13T22:27:10.375ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 251.623089ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "42f190b0-4eaf-427c-80aa-c91517ca4be4", "content-length": "124", "date": "Fri, 13 Dec 2024 22:27:09 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "42f190b0-4eaf-427c-80aa-c91517ca4be4" } file = clients/ddm-admin-client/src/lib.rs:177
1922024-12-13T22:27:10.627ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
1932024-12-13T22:27:10.628ZINFOSledAgent (BootstrapAgentStartup): Failed enable ddm stats (will retry after 326.262157ms) DdmAdminClient = [::1]:8000 err = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "dbbbb3f4-2429-4f90-8c54-0eb03ba1627b", "content-length": "124", "date": "Fri, 13 Dec 2024 22:27:10 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "dbbbb3f4-2429-4f90-8c54-0eb03ba1627b" } file = clients/ddm-admin-client/src/lib.rs:177
1942024-12-13T22:27:10.787ZINFOSledAgent: using '[AddrObject { interface: "net0", name: "ll" }, AddrObject { interface: "net1", name: "ll" }]' as data links for xde driver file = illumos-utils/src/opte/illumos.rs:93 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
1952024-12-13T22:27:10.793ZINFOSledAgent (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:62120 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
1962024-12-13T22:27:10.793ZINFOSledAgent (internal-dns-resolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
1972024-12-13T22:27:10.793ZINFOSledAgent (producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::1]:62120 file = oximeter/producer/src/lib.rs:279 interval = 30s producer_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
1982024-12-13T22:27:10.794ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 213.554041ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
1992024-12-13T22:27:10.795ZINFOSledAgent (VmmReservoirManager): 50% of 102952026112 physical ram = 51476013056 bytes) file = sled-agent/src/vmm_reservoir.rs:267 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
2002024-12-13T22:27:10.795ZINFOSledAgent (VmmReservoirManager): Setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:275 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
2012024-12-13T22:27:10.795ZINFOSledAgent (ServiceManager): sled agent started file = sled-agent/src/services.rs:920 underlay_address = fd00:1122:3344:101::1
2022024-12-13T22:27:10.795ZINFOSledAgent (DnsResolver): new DNS resolver addresses = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = internal-dns/resolver/src/resolver.rs:111
2032024-12-13T22:27:10.956ZINFOSledAgent (BootstrapAgentStartup): Enabling ddm stats DdmAdminClient = [::1]:8000 addr = fd00:1122:3344:101::1 dns_servers = [[fd00:1122:3344:1::1]:53, [fd00:1122:3344:2::1]:53, [fd00:1122:3344:3::1]:53] file = clients/ddm-admin-client/src/lib.rs:163
2042024-12-13T22:27:11.010ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 718.448717ms error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
2052024-12-13T22:27:11.729ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 1.074698422s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
2062024-12-13T22:27:12.805ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 2.653971761s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
2072024-12-13T22:27:13.412ZINFOSledAgent (ServiceManager): Setting up dendrite service file = sled-agent/src/services.rs:2668
2082024-12-13T22:27:13.412ZINFOSledAgent (ServiceManager): Setting up lldpd service file = sled-agent/src/services.rs:2913
2092024-12-13T22:27:13.412ZINFOSledAgent (ServiceManager): Setting up MGS service file = sled-agent/src/services.rs:2538
2102024-12-13T22:27:13.412ZINFOSledAgent (ServiceManager): Setting up wicketd service file = sled-agent/src/services.rs:2585
2112024-12-13T22:27:13.412ZINFOSledAgent (ServiceManager): Setting up mgd service file = sled-agent/src/services.rs:2994
2122024-12-13T22:27:13.412ZINFOSledAgent (ServiceManager): Setting up mg-ddm service file = sled-agent/src/services.rs:3039
2132024-12-13T22:27:13.414ZINFOSledAgent (ServiceManager): Setting up tfport service file = sled-agent/src/services.rs:2838
2142024-12-13T22:27:13.416ZINFOSledAgent (ServiceManager): Setting up Simulated SP service file = sled-agent/src/services.rs:2579
2152024-12-13T22:27:13.416ZINFOSledAgent (ServiceManager): Profile for oxz_switch: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/mgs"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12225'/> <propval type="astring" name="id" value='b25b1e39-ed29-47f2-b89d-b161b54dc3c3'/> <propval type="astring" name="rack_id" value='984ee802-6f4e-4d6b-aed6-da91b38cb72b'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/wicketd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="address" value='[::1]:12226'/> <propval type="astring" name="artifact-address" value='[fdb0:18c0:4d0d:a02a::2]:12227'/> <propval type="astring" name="baseboard-file" value='/opt/oxide/baseboard.json'/> <propval type="astring" name="mgs-address" value='[::1]:12225'/> <propval type="astring" name="nexus-proxy-address" value='[::]:12229'/> <propval type="astring" name="rack-subnet" value='fd00:1122:3344::'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/switch_zone_setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="gz_local_link_addr" value='fe80::8:20ff:fe42:bdbe'/> <property type="astring" name="link_local_links"> <astring_list> <value_node value="tfportrear0_0" /> <value_node value="oxBootstrap0" /> </astring_list> </property> <propval type="astring" name="bootstrap_addr" value='fdb0:18c0:4d0d:a02a::2'/> <propval type="astring" name="bootstrap_vnic" value='oxBootstrap0'/> <propval type="astring" name="baseboard_info" value='{"type":"pc","identifier":"sock","model":"i86pc"}'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/dendrite"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="rack_id" value='984ee802-6f4e-4d6b-aed6-da91b38cb72b'/> <propval type="astring" name="sled_id" value='35eb16d3-adec-4806-a15c-ab7a15d18765'/> <propval type="astring" name="sled_model" value='i86pc'/> <propval type="astring" name="sled_serial" value='sock'/> <propval type="astring" name="sled_revision" value='0'/> <propval type="astring" name="address" value='[::1]:12224'/> <propval type="astring" name="front_ports" value='1'/> <propval type="astring" name="rear_ports" value='1'/> <propval type="astring" name="port_config" value='/opt/oxide/dendrite/misc/softnpu_single_sled_config.toml'/> <propval type="astring" name="mgmt" value='uds'/> <propval type="astring" name="uds_path" value='/opt/softnpu/stuff'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/tfport"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="host" value='[::1]'/> <propval type="astring" name="port" value='12224'/> <propval type="astring" name="flags" value='--sync-only'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/lldpd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="board_rev" value='softnpu_front_1_rear_1'/> <propval type="astring" name="scrimlet_id" value='sock'/> <propval type="astring" name="scrimlet_model" value='i86pc'/> <propval type="astring" name="address" value='[::1]:12230'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/pumpkind"> </service> <service version="1" type="service" name="oxide/mgd"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="sled_uuid" value='35eb16d3-adec-4806-a15c-ab7a15d18765'/> <propval type="astring" name="rack_uuid" value='984ee802-6f4e-4d6b-aed6-da91b38cb72b'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/mg-ddm"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="mode" value='transit'/> <propval type="astring" name="dendrite" value='true'/> <propval type="astring" name="sled_uuid" value='35eb16d3-adec-4806-a15c-ab7a15d18765'/> <propval type="astring" name="rack_uuid" value='984ee802-6f4e-4d6b-aed6-da91b38cb72b'/> <propval type="astring" name="interfaces" value='tfportrear0_0/ll'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/uplink"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
2162024-12-13T22:27:13.416ZINFOSledAgent (ServiceManager): Booting oxz_switch zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_switch
2172024-12-13T22:27:13.765ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 28.213879ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2182024-12-13T22:27:13.962ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 69.888696ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2192024-12-13T22:27:14.140ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 130.483618ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2202024-12-13T22:27:14.317ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 411.309722ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2212024-12-13T22:27:14.930ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 767.406428ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2222024-12-13T22:27:15.462ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 4.149744579s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
2232024-12-13T22:27:15.870ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.28054078s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2242024-12-13T22:27:17.204ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 1.144258598s file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2252024-12-13T22:27:18.369ZWARNSledAgent (ServiceManager): wait for service Some("oxz_switch") failed: Property not found. retry in 814.062466ms file = illumos-utils/src/svc.rs:36 zone = oxz_switch
2262024-12-13T22:27:19.682ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 9.262550984s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
2272024-12-13T22:27:19.688ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/nexus.rs:527 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
2282024-12-13T22:27:19.718ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2292024-12-13T22:27:19.718ZINFOSledAgent (dropshot (Repo Depot)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:12348 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
2302024-12-13T22:27:19.720ZINFOSledAgent (dropshot (SledAgent)): listening file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:327 local_addr = [fd00:1122:3344:101::1]:12345
2312024-12-13T22:27:19.720ZINFOSledAgent (SledAgentMain): Sled Agent loaded; recording configuration file = sled-agent/src/bootstrap/server.rs:432
2322024-12-13T22:27:19.720ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2332024-12-13T22:27:19.720ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2342024-12-13T22:27:19.720ZINFOSledAgent (SledAgentMain): Writing ledger to /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/.sled-agent-request.json.tmp file = common/src/ledger.rs:199
2352024-12-13T22:27:19.720ZINFOSledAgent (HardwareMonitor): Sled Agent Started file = sled-agent/src/hardware_monitor.rs:138
2362024-12-13T22:27:19.720ZINFOSledAgent (HardwareMonitor): Checking current full hardware snapshot disks = {} file = sled-agent/src/hardware_monitor.rs:271 underlay_network_info = Some((fd00:1122:3344:101::2, Some(RackNetworkConfigV2 { rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56 }, infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [PortConfigV2 { routes: [RouteConfig { destination: V4(Ipv4Net { addr: 0.0.0.0, width: 0 }), nexthop: 10.151.2.100, vlan_id: None, rib_priority: None }], addresses: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], switch: Switch0, port: "qsfp0", uplink_port_speed: Speed40G, uplink_port_fec: Some(None), bgp_peers: [], autoneg: false, lldp: None, tx_eq: None }], bgp: [], bfd: [] })))
2372024-12-13T22:27:19.720ZINFOSledAgent (ServiceManager): Ensuring scrimlet services (enabling services) file = sled-agent/src/services.rs:3947
2382024-12-13T22:27:19.720ZINFOSledAgent (SledAgentSprocketsServer): Connection closed file = sled-agent/src/bootstrap/sprockets_server.rs:81 remote_addr = [fdb0:18c0:4d0d:a02a::1]:65160
2392024-12-13T22:27:19.913ZINFOSledAgent (BootstrapAgentRssHandler): Peer agent initialized file = sled-agent/src/bootstrap/rss_handle.rs:117 peer = [fdb0:18c0:4d0d:a02a::1]:12346
2402024-12-13T22:27:19.913ZINFOSledAgent (BootstrapAgentRssHandler): Initialized sled agent file = sled-agent/src/bootstrap/rss_handle.rs:252 target_sled = [fdb0:18c0:4d0d:a02a::1]:12346
2412024-12-13T22:27:19.918ZINFOSledAgent (ServiceManager): Re-enabling running switch zone (new address) file = sled-agent/src/services.rs:4268 new = [fd00:1122:3344:101::2, ::1] old = [::1]
2422024-12-13T22:27:19.918ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists file = sled-agent/src/services.rs:4284
2432024-12-13T22:27:19.919ZINFOSledAgent (ServiceManager): Adding address: Static(V6(Ipv6Network { addr: fd00:1122:3344:101::2, prefix: 64 })) file = illumos-utils/src/running_zone.rs:534 zone = oxz_switch
2442024-12-13T22:27:20.213ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:43114
2452024-12-13T22:27:20.213ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2462024-12-13T22:27:20.213ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
2472024-12-13T22:27:20.213ZINFOSledAgent (ServiceManager): No ledger in /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:188
2482024-12-13T22:27:20.213ZINFOSledAgent (ServiceManager): No ledger in /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:188
2492024-12-13T22:27:20.213ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 347 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:43114 req_id = 348bb3c2-c949-4cfe-a2aa-62b68a7def3c response_code = 200 uri = /inventory
2502024-12-13T22:27:20.244ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:57145
2512024-12-13T22:27:20.244ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:57145 req_id = 5da8540a-dfc1-4939-ba03-a4aa4d3c5086 response_code = 200 uri = /sled-role
2522024-12-13T22:27:20.275ZINFOSledAgent (RSS): attempting to set up sled's storage: OmicronPhysicalDisksConfig { generation: Generation(2), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: cfb3dbc0-9d1b-4054-aba8-5ba597b4550e (physical_disk), pool_id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: d5b38ce4-0eaa-492c-a080-80ecf4c569e2 (physical_disk), pool_id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: ef8cbcec-568b-4440-9052-ee77dbd2ecd6 (physical_disk), pool_id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: b6e90600-59f7-44fa-9a6f-61de1b76aeee (physical_disk), pool_id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 5c1eb23f-57fc-4c39-b88e-637abc36eec4 (physical_disk), pool_id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: c5f1832f-b1f9-44e3-b675-7e3c96f553ab (physical_disk), pool_id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 661617a4-0fb9-4a5a-881b-7571a3cf923f (physical_disk), pool_id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: cb73df23-6286-4e4c-89e7-51792879ba4a (physical_disk), pool_id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 77eee98f-84f8-413a-aa39-f6c4b526d1f7 (physical_disk), pool_id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool) }] } file = sled-agent/src/rack_setup/service.rs:385 sled_address = [fd00:1122:3344:101::1]:12345
2532024-12-13T22:27:20.276ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:50564
2542024-12-13T22:27:20.276ZINFOSledAgent: physical disks ensure file = sled-agent/src/sled_agent.rs:893 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
2552024-12-13T22:27:20.276ZINFOSledAgent (StorageManager): Received OmicronPhysicalDisksEnsure { config: OmicronPhysicalDisksConfig { generation: Generation(2), disks: [OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, id: cfb3dbc0-9d1b-4054-aba8-5ba597b4550e (physical_disk), pool_id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, id: d5b38ce4-0eaa-492c-a080-80ecf4c569e2 (physical_disk), pool_id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, id: ef8cbcec-568b-4440-9052-ee77dbd2ecd6 (physical_disk), pool_id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, id: b6e90600-59f7-44fa-9a6f-61de1b76aeee (physical_disk), pool_id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, id: 5c1eb23f-57fc-4c39-b88e-637abc36eec4 (physical_disk), pool_id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, id: c5f1832f-b1f9-44e3-b675-7e3c96f553ab (physical_disk), pool_id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, id: 661617a4-0fb9-4a5a-881b-7571a3cf923f (physical_disk), pool_id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, id: cb73df23-6286-4e4c-89e7-51792879ba4a (physical_disk), pool_id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool) }, OmicronPhysicalDiskConfig { identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, id: 77eee98f-84f8-413a-aa39-f6c4b526d1f7 (physical_disk), pool_id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool) }] }, tx: ... } file = sled-storage/src/manager.rs:603
2562024-12-13T22:27:20.276ZINFOSledAgent (StorageManager): No ledger in /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2572024-12-13T22:27:20.276ZINFOSledAgent (StorageManager): No ledger in /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-physical-disks.json file = common/src/ledger.rs:188 request = omicron_physical_disks_ensure
2582024-12-13T22:27:20.276ZINFOSledAgent (StorageManager): No previously-stored 'requested disks', creating new ledger file = sled-storage/src/manager.rs:1171 request = omicron_physical_disks_ensure
2592024-12-13T22:27:20.276ZINFOSledAgent (StorageResources): Synchronizing disk managment file = sled-storage/src/resources.rs:302
2602024-12-13T22:27:20.276ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:342
2612024-12-13T22:27:20.276ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2622024-12-13T22:27:20.276ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } path = /scratch/u2_0.vdev
2632024-12-13T22:27:20.430ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_0.vdev file = sled-hardware/src/disk.rs:367
2642024-12-13T22:27:20.430ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool)
2652024-12-13T22:27:20.888ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/probe_manager.rs:231 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
2662024-12-13T22:27:20.888ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17993106840727764337 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
2672024-12-13T22:27:20.888ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17332408252662968763 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
2682024-12-13T22:27:21.115ZINFOSledAgent (VmmReservoirManager): Finished setting reservoir size to 49090 MiB bytes file = sled-agent/src/vmm_reservoir.rs:279 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
2692024-12-13T22:27:21.164ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }
2702024-12-13T22:27:21.172ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/dataset.rs:240
2712024-12-13T22:27:21.172ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:242
2722024-12-13T22:27:21.173ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:246
2732024-12-13T22:27:21.173ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } epoch = 0 file = sled-storage/src/dataset.rs:248
2742024-12-13T22:27:21.173ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2752024-12-13T22:27:21.173ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key")
2762024-12-13T22:27:21.173ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
2772024-12-13T22:27:21.206ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-0-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2782024-12-13T22:27:21.214ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/zone file = sled-storage/src/dataset.rs:303
2792024-12-13T22:27:21.394ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5 file = sled-storage/src/dataset.rs:389
2802024-12-13T22:27:21.402ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:440
2812024-12-13T22:27:21.402ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" } file = sled-storage/src/resources.rs:357
2822024-12-13T22:27:21.402ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:342
2832024-12-13T22:27:21.402ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
2842024-12-13T22:27:21.402ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } path = /scratch/u2_1.vdev
2852024-12-13T22:27:21.413ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_1.vdev file = sled-hardware/src/disk.rs:367
2862024-12-13T22:27:21.413ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool)
2872024-12-13T22:27:21.510ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }
2882024-12-13T22:27:21.519ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/dataset.rs:240
2892024-12-13T22:27:21.519ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:242
2902024-12-13T22:27:21.519ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:246
2912024-12-13T22:27:21.519ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } epoch = 0 file = sled-storage/src/dataset.rs:248
2922024-12-13T22:27:21.519ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2932024-12-13T22:27:21.519ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key")
2942024-12-13T22:27:21.519ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
2952024-12-13T22:27:21.555ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-1-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
2962024-12-13T22:27:21.563ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f/crypt/zone file = sled-storage/src/dataset.rs:303
2972024-12-13T22:27:21.740ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f file = sled-storage/src/dataset.rs:389
2982024-12-13T22:27:21.748ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:440
2992024-12-13T22:27:21.748ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" } file = sled-storage/src/resources.rs:357
3002024-12-13T22:27:21.748ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:342
3012024-12-13T22:27:21.748ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3022024-12-13T22:27:21.748ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } path = /scratch/u2_2.vdev
3032024-12-13T22:27:21.758ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_2.vdev file = sled-hardware/src/disk.rs:367
3042024-12-13T22:27:21.758ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool)
3052024-12-13T22:27:21.853ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }
3062024-12-13T22:27:21.861ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/dataset.rs:240
3072024-12-13T22:27:21.861ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:242
3082024-12-13T22:27:21.861ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:246
3092024-12-13T22:27:21.861ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } epoch = 0 file = sled-storage/src/dataset.rs:248
3102024-12-13T22:27:21.861ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3112024-12-13T22:27:21.861ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key")
3122024-12-13T22:27:21.861ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3132024-12-13T22:27:21.890ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/probe_manager.rs:231 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
3142024-12-13T22:27:21.897ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-2-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3152024-12-13T22:27:21.906ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_9e30dd3c-3e72-4b01-8f5f-662db99dc37a/crypt/zone file = sled-storage/src/dataset.rs:303
3162024-12-13T22:27:22.087ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_9e30dd3c-3e72-4b01-8f5f-662db99dc37a file = sled-storage/src/dataset.rs:389
3172024-12-13T22:27:22.095ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:440
3182024-12-13T22:27:22.095ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" } file = sled-storage/src/resources.rs:357
3192024-12-13T22:27:22.095ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:342
3202024-12-13T22:27:22.095ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3212024-12-13T22:27:22.095ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } path = /scratch/u2_3.vdev
3222024-12-13T22:27:22.105ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_3.vdev file = sled-hardware/src/disk.rs:367
3232024-12-13T22:27:22.105ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool)
3242024-12-13T22:27:22.204ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }
3252024-12-13T22:27:22.213ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/dataset.rs:240
3262024-12-13T22:27:22.213ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:242
3272024-12-13T22:27:22.213ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:246
3282024-12-13T22:27:22.213ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } epoch = 0 file = sled-storage/src/dataset.rs:248
3292024-12-13T22:27:22.213ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3302024-12-13T22:27:22.213ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key")
3312024-12-13T22:27:22.213ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3322024-12-13T22:27:22.249ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-3-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3332024-12-13T22:27:22.257ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_8159e296-b0fc-4958-9b6f-9386d6e003fa/crypt/zone file = sled-storage/src/dataset.rs:303
3342024-12-13T22:27:22.435ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_8159e296-b0fc-4958-9b6f-9386d6e003fa file = sled-storage/src/dataset.rs:389
3352024-12-13T22:27:22.444ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:440
3362024-12-13T22:27:22.444ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" } file = sled-storage/src/resources.rs:357
3372024-12-13T22:27:22.444ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:342
3382024-12-13T22:27:22.444ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3392024-12-13T22:27:22.444ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } path = /scratch/u2_4.vdev
3402024-12-13T22:27:22.455ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_4.vdev file = sled-hardware/src/disk.rs:367
3412024-12-13T22:27:22.455ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = fb594314-d689-4df6-8176-e09675ab8f03 (zpool)
3422024-12-13T22:27:22.553ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }
3432024-12-13T22:27:22.562ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/dataset.rs:240
3442024-12-13T22:27:22.562ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:242
3452024-12-13T22:27:22.562ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:246
3462024-12-13T22:27:22.562ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } epoch = 0 file = sled-storage/src/dataset.rs:248
3472024-12-13T22:27:22.562ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3482024-12-13T22:27:22.562ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key")
3492024-12-13T22:27:22.562ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3502024-12-13T22:27:22.597ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-4-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3512024-12-13T22:27:22.605ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_fb594314-d689-4df6-8176-e09675ab8f03/crypt/zone file = sled-storage/src/dataset.rs:303
3522024-12-13T22:27:22.782ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_fb594314-d689-4df6-8176-e09675ab8f03 file = sled-storage/src/dataset.rs:389
3532024-12-13T22:27:22.791ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:440
3542024-12-13T22:27:22.791ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" } file = sled-storage/src/resources.rs:357
3552024-12-13T22:27:22.791ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:342
3562024-12-13T22:27:22.791ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3572024-12-13T22:27:22.791ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } path = /scratch/u2_5.vdev
3582024-12-13T22:27:22.801ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_5.vdev file = sled-hardware/src/disk.rs:367
3592024-12-13T22:27:22.801ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool)
3602024-12-13T22:27:22.896ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }
3612024-12-13T22:27:22.905ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/dataset.rs:240
3622024-12-13T22:27:22.905ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:242
3632024-12-13T22:27:22.905ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:246
3642024-12-13T22:27:22.905ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } epoch = 0 file = sled-storage/src/dataset.rs:248
3652024-12-13T22:27:22.905ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3662024-12-13T22:27:22.905ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key")
3672024-12-13T22:27:22.905ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3682024-12-13T22:27:22.909ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/probe_manager.rs:231 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
3692024-12-13T22:27:22.942ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-5-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3702024-12-13T22:27:22.951ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_5bc1d750-0ca6-4a08-bf34-a01bf34cccef/crypt/zone file = sled-storage/src/dataset.rs:303
3712024-12-13T22:27:23.133ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_5bc1d750-0ca6-4a08-bf34-a01bf34cccef file = sled-storage/src/dataset.rs:389
3722024-12-13T22:27:23.141ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:440
3732024-12-13T22:27:23.141ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" } file = sled-storage/src/resources.rs:357
3742024-12-13T22:27:23.141ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:342
3752024-12-13T22:27:23.141ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3762024-12-13T22:27:23.141ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } path = /scratch/u2_6.vdev
3772024-12-13T22:27:23.151ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_6.vdev file = sled-hardware/src/disk.rs:367
3782024-12-13T22:27:23.151ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool)
3792024-12-13T22:27:23.254ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }
3802024-12-13T22:27:23.264ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/dataset.rs:240
3812024-12-13T22:27:23.264ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:242
3822024-12-13T22:27:23.264ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:246
3832024-12-13T22:27:23.264ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } epoch = 0 file = sled-storage/src/dataset.rs:248
3842024-12-13T22:27:23.264ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3852024-12-13T22:27:23.264ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key")
3862024-12-13T22:27:23.264ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
3872024-12-13T22:27:23.303ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-6-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
3882024-12-13T22:27:23.310ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_b80657b8-dd44-496f-8b60-edeffe6ea0c4/crypt/zone file = sled-storage/src/dataset.rs:303
3892024-12-13T22:27:23.487ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_b80657b8-dd44-496f-8b60-edeffe6ea0c4 file = sled-storage/src/dataset.rs:389
3902024-12-13T22:27:23.494ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:440
3912024-12-13T22:27:23.495ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" } file = sled-storage/src/resources.rs:357
3922024-12-13T22:27:23.495ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:342
3932024-12-13T22:27:23.495ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
3942024-12-13T22:27:23.495ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } path = /scratch/u2_7.vdev
3952024-12-13T22:27:23.505ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_7.vdev file = sled-hardware/src/disk.rs:367
3962024-12-13T22:27:23.505ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool)
3972024-12-13T22:27:23.651ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External }
3982024-12-13T22:27:23.660ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/dataset.rs:240
3992024-12-13T22:27:23.660ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:242
4002024-12-13T22:27:23.660ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:246
4012024-12-13T22:27:23.660ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } epoch = 0 file = sled-storage/src/dataset.rs:248
4022024-12-13T22:27:23.660ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4032024-12-13T22:27:23.660ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key")
4042024-12-13T22:27:23.660ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
4052024-12-13T22:27:23.823ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-7-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4062024-12-13T22:27:23.831ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_35c96663-c574-4d07-a692-3eb8a2a79302/crypt/zone file = sled-storage/src/dataset.rs:303
4072024-12-13T22:27:23.911ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/probe_manager.rs:231 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4082024-12-13T22:27:24.716ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_35c96663-c574-4d07-a692-3eb8a2a79302 file = sled-storage/src/dataset.rs:389
4092024-12-13T22:27:24.725ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:440
4102024-12-13T22:27:24.725ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" } file = sled-storage/src/resources.rs:357
4112024-12-13T22:27:24.725ZINFOSledAgent (StorageResources): Managing disk disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:342
4122024-12-13T22:27:24.725ZINFOSledAgent (StorageResources): Invoking Disk::new on an unmanaged disk file = sled-storage/src/resources.rs:419
4132024-12-13T22:27:24.725ZINFOSledAgent (StorageResources): Invoking SyntheticDisk::new file = sled-storage/src/disk.rs:65 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } path = /scratch/u2_8.vdev
4142024-12-13T22:27:24.735ZINFOSledAgent (StorageResources): GPT exists without Zpool: formatting zpool at /scratch/u2_8.vdev file = sled-hardware/src/disk.rs:367
4152024-12-13T22:27:24.735ZINFOSledAgent (StorageResources): Formatting zpool with requested ID file = sled-hardware/src/disk.rs:373 id = ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool)
4162024-12-13T22:27:26.434ZINFOSledAgent (StorageResources): Ensuring zpool has datasets disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:196 zpool = ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }
4172024-12-13T22:27:26.443ZINFOSledAgent (StorageResources): Loading latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/dataset.rs:240
4182024-12-13T22:27:26.443ZINFOSledAgent (StorageResources): Loaded latest secret disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:242
4192024-12-13T22:27:26.443ZINFOSledAgent (StorageResources): Retrieving key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:246
4202024-12-13T22:27:26.443ZINFOSledAgent (StorageResources): Got key disk_id = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } epoch = 0 file = sled-storage/src/dataset.rs:248
4212024-12-13T22:27:26.444ZINFOSledAgent (StorageResources): About to create keyfile file = sled-storage/src/keyfile.rs:30 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4222024-12-13T22:27:26.444ZINFOSledAgent (StorageResources): Created keyfile file = sled-storage/src/keyfile.rs:34 path = Keypath("/var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key")
4232024-12-13T22:27:26.444ZINFOSledAgent (StorageResources): Ensuring encrypted filesystem: crypt for epoch 0 file = sled-storage/src/dataset.rs:260
4242024-12-13T22:27:26.447ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/probe_manager.rs:231 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4252024-12-13T22:27:26.586ZINFOSledAgent (StorageResources): Zeroed and unlinked keyfile /var/run/oxide/synthetic-vendor-synthetic-serial-8-synthetic-model-U2-zfs-aes-256-gcm.key file = sled-storage/src/keyfile.rs:53
4262024-12-13T22:27:26.596ZINFOSledAgent (StorageResources): Automatically destroying dataset: oxp_ffd405c4-ad68-4401-9d01-3a9d70b7775f/crypt/zone file = sled-storage/src/dataset.rs:303
4272024-12-13T22:27:27.450ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/probe_manager.rs:231 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4282024-12-13T22:27:27.556ZINFOSledAgent (StorageResources): Looking for unencrypted datasets in oxp_ffd405c4-ad68-4401-9d01-3a9d70b7775f file = sled-storage/src/dataset.rs:389
4292024-12-13T22:27:27.564ZINFOSledAgent (StorageResources): Disk::new completed successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:440
4302024-12-13T22:27:27.564ZINFOSledAgent (StorageResources): Disk management started successfully disk_identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" } file = sled-storage/src/resources.rs:357
4312024-12-13T22:27:27.564ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4322024-12-13T22:27:27.564ZINFOSledAgent (StorageMonitor): Received storage manager update disks = AllDisks { generation: Generation(2), inner: AllDisksInner { values: {DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }, variant: M2, slot: 1024, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 6c407cb8-0013-4482-9b28-b5bd52e5b10f (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }: ImplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/m2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }, variant: M2, slot: 1025, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: dbb4b417-a6b8-4743-be7a-9642ba91ec5c (zpool), kind: Internal } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_0.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }, variant: U2, slot: 1026, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_1.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }, variant: U2, slot: 1027, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_2.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }, variant: U2, slot: 1028, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_3.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }, variant: U2, slot: 1029, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_4.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }, variant: U2, slot: 1030, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_5.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }, variant: U2, slot: 1031, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_6.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }, variant: U2, slot: 1032, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_7.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }, variant: U2, slot: 1033, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External } })), DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }: ExplicitlyManaged(Synthetic(SyntheticDisk { raw: RawSyntheticDisk { path: "/scratch/u2_8.vdev", identity: DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }, variant: U2, slot: 1034, firmware: DiskFirmware { active_slot: 1, next_active_slot: None, slot1_read_only: true, number_of_slots: 1, slot_firmware_versions: [Some("SYNTH1")] } }, zpool_name: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External } }))} }, mount_config: MountConfig { root: "/", synthetic_disk_root: "/var/tmp" } } file = sled-agent/src/storage_monitor.rs:93
4332024-12-13T22:27:27.564ZINFOSledAgent (DumpSetup-worker): Updated view of disks core_datasets = 0 debug_datasets = 0 dump_slices = 0 file = sled-agent/src/dump_setup.rs:580
4342024-12-13T22:27:27.564ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
4352024-12-13T22:27:27.564ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/.omicron-physical-disks.json.tmp file = common/src/ledger.rs:199 request = omicron_physical_disks_ensure
4362024-12-13T22:27:27.565ZINFOSledAgent: physical disks ensure: Updated storage file = sled-agent/src/sled_agent.rs:897 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4372024-12-13T22:27:27.565ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
4382024-12-13T22:27:27.565ZINFOSledAgent: physical disks ensure: Propagating new generation of disks file = sled-agent/src/sled_agent.rs:923 generation = Generation(2) sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4392024-12-13T22:27:27.565ZINFOSledAgent: physical disks ensure: Updated storage monitor file = sled-agent/src/sled_agent.rs:928 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4402024-12-13T22:27:27.565ZINFOSledAgent: physical disks ensure: Updated zone bundler file = sled-agent/src/sled_agent.rs:933 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4412024-12-13T22:27:27.565ZINFOSledAgent (ProbeManager): use_only_these_disks: Processing new request file = sled-agent/src/probe_manager.rs:122 gen = Generation(2) sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4422024-12-13T22:27:27.565ZINFOSledAgent: physical disks ensure: Updated probes file = sled-agent/src/sled_agent.rs:939 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4432024-12-13T22:27:27.565ZINFOSledAgent (InstanceManager): use_only_these_disks: Processing new request file = sled-agent/src/instance_manager.rs:799 gen = Generation(2)
4442024-12-13T22:27:27.565ZINFOSledAgent: physical disks ensure: Updated instances file = sled-agent/src/sled_agent.rs:944 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4452024-12-13T22:27:27.565ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7289317 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:50564 req_id = bbdbdff0-a1a3-4f35-b4f2-95e1c0edd860 response_code = 200 uri = /omicron-physical-disks
4462024-12-13T22:27:27.597ZINFOSledAgent (RSS): attempting to set up sled's datasets: DatasetsConfig { generation: Generation(1), datasets: {056076d5-1f2a-442b-9205-5038c0ae0212 (dataset): DatasetConfig { id: 056076d5-1f2a-442b-9205-5038c0ae0212 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0bf7df9d-008f-4dae-b31c-0290b598034b (dataset): DatasetConfig { id: 0bf7df9d-008f-4dae-b31c-0290b598034b (dataset), name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1570e3f6-27ad-4f57-ad82-7ac1f68a77db (dataset): DatasetConfig { id: 1570e3f6-27ad-4f57-ad82-7ac1f68a77db (dataset), name: DatasetName { pool_name: ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1724c981-d720-44c5-a8a9-54bbd919d0de (dataset): DatasetConfig { id: 1724c981-d720-44c5-a8a9-54bbd919d0de (dataset), name: DatasetName { pool_name: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 1b1d4e16-a972-4486-8b3d-d9203dba4809 (dataset): DatasetConfig { id: 1b1d4e16-a972-4486-8b3d-d9203dba4809 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1ff0cdbb-086b-45b7-8702-dd68a4672db1 (dataset): DatasetConfig { id: 1ff0cdbb-086b-45b7-8702-dd68a4672db1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 22b8db67-2aae-4af1-b8da-2876093092c9 (dataset): DatasetConfig { id: 22b8db67-2aae-4af1-b8da-2876093092c9 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: TransientZone { name: "oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 258cc09a-28b1-4ad9-a692-456a54c73e58 (dataset): DatasetConfig { id: 258cc09a-28b1-4ad9-a692-456a54c73e58 (dataset), name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2d30d5ea-9628-4631-9179-f7581f2a3980 (dataset): DatasetConfig { id: 2d30d5ea-9628-4631-9179-f7581f2a3980 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 364ed6e2-2953-4955-b9bd-39ebfe40b2c1 (dataset): DatasetConfig { id: 364ed6e2-2953-4955-b9bd-39ebfe40b2c1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 374407d7-b75e-4015-b0c5-5c8021ccc5cb (dataset): DatasetConfig { id: 374407d7-b75e-4015-b0c5-5c8021ccc5cb (dataset), name: DatasetName { pool_name: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 374e3d59-e95d-46e4-b6cd-32630f0b8032 (dataset): DatasetConfig { id: 374e3d59-e95d-46e4-b6cd-32630f0b8032 (dataset), name: DatasetName { pool_name: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3a7126b5-59cd-42c5-ae93-04857ab596b0 (dataset): DatasetConfig { id: 3a7126b5-59cd-42c5-ae93-04857ab596b0 (dataset), name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3b598c61-c094-4e0d-a35c-afe42631417f (dataset): DatasetConfig { id: 3b598c61-c094-4e0d-a35c-afe42631417f (dataset), name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4358585d-c7b8-4c91-8e06-b970bf065361 (dataset): DatasetConfig { id: 4358585d-c7b8-4c91-8e06-b970bf065361 (dataset), name: DatasetName { pool_name: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, kind: TransientZone { name: "oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 49543e77-be7c-4c39-b7cb-cbf1306b1193 (dataset): DatasetConfig { id: 49543e77-be7c-4c39-b7cb-cbf1306b1193 (dataset), name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 49bf14dd-3828-4336-8285-801327ad236b (dataset): DatasetConfig { id: 49bf14dd-3828-4336-8285-801327ad236b (dataset), name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4a5c2333-ef07-424c-b520-9b1882bfe156 (dataset): DatasetConfig { id: 4a5c2333-ef07-424c-b520-9b1882bfe156 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 4b3475ff-005d-4871-bf89-b72a73c348f0 (dataset): DatasetConfig { id: 4b3475ff-005d-4871-bf89-b72a73c348f0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4d5de073-7e5a-42c4-a943-14d888fb032c (dataset): DatasetConfig { id: 4d5de073-7e5a-42c4-a943-14d888fb032c (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4f754089-09f9-40bd-8f5e-0a4a145a719c (dataset): DatasetConfig { id: 4f754089-09f9-40bd-8f5e-0a4a145a719c (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 52bc7a57-09f4-49d2-a229-e75df6b609c5 (dataset): DatasetConfig { id: 52bc7a57-09f4-49d2-a229-e75df6b609c5 (dataset), name: DatasetName { pool_name: ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5302abd9-e922-4ec1-a16f-17b8e3f973c2 (dataset): DatasetConfig { id: 5302abd9-e922-4ec1-a16f-17b8e3f973c2 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 63091da1-be31-4c6c-b7a4-2bfd0051696c (dataset): DatasetConfig { id: 63091da1-be31-4c6c-b7a4-2bfd0051696c (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 690b61b7-78ff-4954-b033-495b50b50fa5 (dataset): DatasetConfig { id: 690b61b7-78ff-4954-b033-495b50b50fa5 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 69ce4561-5e4f-4a05-8e43-e345524fc356 (dataset): DatasetConfig { id: 69ce4561-5e4f-4a05-8e43-e345524fc356 (dataset), name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 69e630a9-04f3-4564-a298-f686ad413941 (dataset): DatasetConfig { id: 69e630a9-04f3-4564-a298-f686ad413941 (dataset), name: DatasetName { pool_name: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 6ea9f107-472c-4fc7-92fd-9381ffc695e3 (dataset): DatasetConfig { id: 6ea9f107-472c-4fc7-92fd-9381ffc695e3 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 7b9a0de1-dbbb-4f30-9914-d61435b5a856 (dataset): DatasetConfig { id: 7b9a0de1-dbbb-4f30-9914-d61435b5a856 (dataset), name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 83a4be6f-f4ff-4d4c-904e-25eee6a15c8d (dataset): DatasetConfig { id: 83a4be6f-f4ff-4d4c-904e-25eee6a15c8d (dataset), name: DatasetName { pool_name: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 84d741d2-2a91-4ec3-892a-db99759a5cc5 (dataset): DatasetConfig { id: 84d741d2-2a91-4ec3-892a-db99759a5cc5 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 858f902a-21d2-4c13-b3d2-8c4ee62ba67b (dataset): DatasetConfig { id: 858f902a-21d2-4c13-b3d2-8c4ee62ba67b (dataset), name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 885b9e27-54b2-4713-b093-6d874ca1349c (dataset): DatasetConfig { id: 885b9e27-54b2-4713-b093-6d874ca1349c (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 88906eca-0959-4108-a236-625d08791368 (dataset): DatasetConfig { id: 88906eca-0959-4108-a236-625d08791368 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8c27bc2a-bdd2-4313-97c9-79f3db27f586 (dataset): DatasetConfig { id: 8c27bc2a-bdd2-4313-97c9-79f3db27f586 (dataset), name: DatasetName { pool_name: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8dae09fe-00c8-46ae-b5c3-72be58ce0516 (dataset): DatasetConfig { id: 8dae09fe-00c8-46ae-b5c3-72be58ce0516 (dataset), name: DatasetName { pool_name: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8e24fefc-519d-4f3a-af94-8e0a70bea3d7 (dataset): DatasetConfig { id: 8e24fefc-519d-4f3a-af94-8e0a70bea3d7 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9247aa19-6e12-4c4c-bf8b-befd2a265431 (dataset): DatasetConfig { id: 9247aa19-6e12-4c4c-bf8b-befd2a265431 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 951a6c80-3a95-48f7-b916-19ac4f14e8e0 (dataset): DatasetConfig { id: 951a6c80-3a95-48f7-b916-19ac4f14e8e0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 966682e0-68c3-4075-abba-da36520e21df (dataset): DatasetConfig { id: 966682e0-68c3-4075-abba-da36520e21df (dataset), name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9bd3ca48-034a-4de7-aa7f-e05ae7805f3c (dataset): DatasetConfig { id: 9bd3ca48-034a-4de7-aa7f-e05ae7805f3c (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9c792e48-71c2-495a-af3a-73fcbf853594 (dataset): DatasetConfig { id: 9c792e48-71c2-495a-af3a-73fcbf853594 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, a2f5fdb5-1433-4caa-b567-19c860c92bb2 (dataset): DatasetConfig { id: a2f5fdb5-1433-4caa-b567-19c860c92bb2 (dataset), name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, afab4ac7-d3ef-4901-a413-20d6cf347bca (dataset): DatasetConfig { id: afab4ac7-d3ef-4901-a413-20d6cf347bca (dataset), name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b205f29c-82b5-4f47-9ce8-041d18372567 (dataset): DatasetConfig { id: b205f29c-82b5-4f47-9ce8-041d18372567 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Clickhouse }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, babc237b-bd7a-4272-8cb5-b2cabd5aed75 (dataset): DatasetConfig { id: babc237b-bd7a-4272-8cb5-b2cabd5aed75 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, bb12f4bf-3249-411f-9b29-011aa90db3a8 (dataset): DatasetConfig { id: bb12f4bf-3249-411f-9b29-011aa90db3a8 (dataset), name: DatasetName { pool_name: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, bd6b6802-378b-475d-abba-948bfed76657 (dataset): DatasetConfig { id: bd6b6802-378b-475d-abba-948bfed76657 (dataset), name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, c032111e-dfc8-4bc0-a85b-b1481e666919 (dataset): DatasetConfig { id: c032111e-dfc8-4bc0-a85b-b1481e666919 (dataset), name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ce7d683a-d202-43c1-8f0c-a51400165e9a (dataset): DatasetConfig { id: ce7d683a-d202-43c1-8f0c-a51400165e9a (dataset), name: DatasetName { pool_name: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ce8ab814-584e-4716-922a-431c20480d22 (dataset): DatasetConfig { id: ce8ab814-584e-4716-922a-431c20480d22 (dataset), name: DatasetName { pool_name: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ce974e7d-36ab-4da1-b508-68af2c74f0d4 (dataset): DatasetConfig { id: ce974e7d-36ab-4da1-b508-68af2c74f0d4 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d55c4b03-f011-4fb9-bf0f-5d707bcdf313 (dataset): DatasetConfig { id: d55c4b03-f011-4fb9-bf0f-5d707bcdf313 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d921c93e-3b4f-4cca-89a8-d2a875645434 (dataset): DatasetConfig { id: d921c93e-3b4f-4cca-89a8-d2a875645434 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: TransientZone { name: "oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e07f87c3-9653-4896-9bed-29768c167e06 (dataset): DatasetConfig { id: e07f87c3-9653-4896-9bed-29768c167e06 (dataset), name: DatasetName { pool_name: ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, e21216a5-4440-4566-92e8-5a940464548d (dataset): DatasetConfig { id: e21216a5-4440-4566-92e8-5a940464548d (dataset), name: DatasetName { pool_name: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e25a0d38-c737-438c-a76b-38da18d40ebc (dataset): DatasetConfig { id: e25a0d38-c737-438c-a76b-38da18d40ebc (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e64d431c-80f8-459a-8f90-046dbb2eb3ba (dataset): DatasetConfig { id: e64d431c-80f8-459a-8f90-046dbb2eb3ba (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ed989601-2478-45af-a6fd-32a0620e5978 (dataset): DatasetConfig { id: ed989601-2478-45af-a6fd-32a0620e5978 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f069dbd2-41f5-4bef-8984-2cfbf8404905 (dataset): DatasetConfig { id: f069dbd2-41f5-4bef-8984-2cfbf8404905 (dataset), name: DatasetName { pool_name: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f22a73bf-4f31-4b69-8a91-67ce66648c5c (dataset): DatasetConfig { id: f22a73bf-4f31-4b69-8a91-67ce66648c5c (dataset), name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f4bc7c77-6eaf-4898-b6d9-be3148894069 (dataset): DatasetConfig { id: f4bc7c77-6eaf-4898-b6d9-be3148894069 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f57c966d-590d-4a66-b67b-207616cadf28 (dataset): DatasetConfig { id: f57c966d-590d-4a66-b67b-207616cadf28 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f662aea4-133d-4e18-8adf-7d23948c57df (dataset): DatasetConfig { id: f662aea4-133d-4e18-8adf-7d23948c57df (dataset), name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f736beca-68a0-4673-9319-f7ce56a49190 (dataset): DatasetConfig { id: f736beca-68a0-4673-9319-f7ce56a49190 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f7f2c66a-51cf-4098-97b9-a80ecdb5dbb1 (dataset): DatasetConfig { id: f7f2c66a-51cf-4098-97b9-a80ecdb5dbb1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }} } file = sled-agent/src/rack_setup/service.rs:460 sled_address = [fd00:1122:3344:101::1]:12345
4472024-12-13T22:27:27.597ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:63873
4482024-12-13T22:27:27.598ZINFOSledAgent: datasets ensure file = sled-agent/src/sled_agent.rs:859 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4492024-12-13T22:27:27.598ZINFOSledAgent (StorageManager): Received DatasetsEnsure { config: DatasetsConfig { generation: Generation(1), datasets: {056076d5-1f2a-442b-9205-5038c0ae0212 (dataset): DatasetConfig { id: 056076d5-1f2a-442b-9205-5038c0ae0212 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 0bf7df9d-008f-4dae-b31c-0290b598034b (dataset): DatasetConfig { id: 0bf7df9d-008f-4dae-b31c-0290b598034b (dataset), name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1570e3f6-27ad-4f57-ad82-7ac1f68a77db (dataset): DatasetConfig { id: 1570e3f6-27ad-4f57-ad82-7ac1f68a77db (dataset), name: DatasetName { pool_name: ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1724c981-d720-44c5-a8a9-54bbd919d0de (dataset): DatasetConfig { id: 1724c981-d720-44c5-a8a9-54bbd919d0de (dataset), name: DatasetName { pool_name: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 1b1d4e16-a972-4486-8b3d-d9203dba4809 (dataset): DatasetConfig { id: 1b1d4e16-a972-4486-8b3d-d9203dba4809 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 1ff0cdbb-086b-45b7-8702-dd68a4672db1 (dataset): DatasetConfig { id: 1ff0cdbb-086b-45b7-8702-dd68a4672db1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 22b8db67-2aae-4af1-b8da-2876093092c9 (dataset): DatasetConfig { id: 22b8db67-2aae-4af1-b8da-2876093092c9 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: TransientZone { name: "oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 258cc09a-28b1-4ad9-a692-456a54c73e58 (dataset): DatasetConfig { id: 258cc09a-28b1-4ad9-a692-456a54c73e58 (dataset), name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 2d30d5ea-9628-4631-9179-f7581f2a3980 (dataset): DatasetConfig { id: 2d30d5ea-9628-4631-9179-f7581f2a3980 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 364ed6e2-2953-4955-b9bd-39ebfe40b2c1 (dataset): DatasetConfig { id: 364ed6e2-2953-4955-b9bd-39ebfe40b2c1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 374407d7-b75e-4015-b0c5-5c8021ccc5cb (dataset): DatasetConfig { id: 374407d7-b75e-4015-b0c5-5c8021ccc5cb (dataset), name: DatasetName { pool_name: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 374e3d59-e95d-46e4-b6cd-32630f0b8032 (dataset): DatasetConfig { id: 374e3d59-e95d-46e4-b6cd-32630f0b8032 (dataset), name: DatasetName { pool_name: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3a7126b5-59cd-42c5-ae93-04857ab596b0 (dataset): DatasetConfig { id: 3a7126b5-59cd-42c5-ae93-04857ab596b0 (dataset), name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 3b598c61-c094-4e0d-a35c-afe42631417f (dataset): DatasetConfig { id: 3b598c61-c094-4e0d-a35c-afe42631417f (dataset), name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4358585d-c7b8-4c91-8e06-b970bf065361 (dataset): DatasetConfig { id: 4358585d-c7b8-4c91-8e06-b970bf065361 (dataset), name: DatasetName { pool_name: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, kind: TransientZone { name: "oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 49543e77-be7c-4c39-b7cb-cbf1306b1193 (dataset): DatasetConfig { id: 49543e77-be7c-4c39-b7cb-cbf1306b1193 (dataset), name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 49bf14dd-3828-4336-8285-801327ad236b (dataset): DatasetConfig { id: 49bf14dd-3828-4336-8285-801327ad236b (dataset), name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4a5c2333-ef07-424c-b520-9b1882bfe156 (dataset): DatasetConfig { id: 4a5c2333-ef07-424c-b520-9b1882bfe156 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 4b3475ff-005d-4871-bf89-b72a73c348f0 (dataset): DatasetConfig { id: 4b3475ff-005d-4871-bf89-b72a73c348f0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4d5de073-7e5a-42c4-a943-14d888fb032c (dataset): DatasetConfig { id: 4d5de073-7e5a-42c4-a943-14d888fb032c (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 4f754089-09f9-40bd-8f5e-0a4a145a719c (dataset): DatasetConfig { id: 4f754089-09f9-40bd-8f5e-0a4a145a719c (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 52bc7a57-09f4-49d2-a229-e75df6b609c5 (dataset): DatasetConfig { id: 52bc7a57-09f4-49d2-a229-e75df6b609c5 (dataset), name: DatasetName { pool_name: ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 5302abd9-e922-4ec1-a16f-17b8e3f973c2 (dataset): DatasetConfig { id: 5302abd9-e922-4ec1-a16f-17b8e3f973c2 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 63091da1-be31-4c6c-b7a4-2bfd0051696c (dataset): DatasetConfig { id: 63091da1-be31-4c6c-b7a4-2bfd0051696c (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 690b61b7-78ff-4954-b033-495b50b50fa5 (dataset): DatasetConfig { id: 690b61b7-78ff-4954-b033-495b50b50fa5 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 69ce4561-5e4f-4a05-8e43-e345524fc356 (dataset): DatasetConfig { id: 69ce4561-5e4f-4a05-8e43-e345524fc356 (dataset), name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 69e630a9-04f3-4564-a298-f686ad413941 (dataset): DatasetConfig { id: 69e630a9-04f3-4564-a298-f686ad413941 (dataset), name: DatasetName { pool_name: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 6ea9f107-472c-4fc7-92fd-9381ffc695e3 (dataset): DatasetConfig { id: 6ea9f107-472c-4fc7-92fd-9381ffc695e3 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, 7b9a0de1-dbbb-4f30-9914-d61435b5a856 (dataset): DatasetConfig { id: 7b9a0de1-dbbb-4f30-9914-d61435b5a856 (dataset), name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 83a4be6f-f4ff-4d4c-904e-25eee6a15c8d (dataset): DatasetConfig { id: 83a4be6f-f4ff-4d4c-904e-25eee6a15c8d (dataset), name: DatasetName { pool_name: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 84d741d2-2a91-4ec3-892a-db99759a5cc5 (dataset): DatasetConfig { id: 84d741d2-2a91-4ec3-892a-db99759a5cc5 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 858f902a-21d2-4c13-b3d2-8c4ee62ba67b (dataset): DatasetConfig { id: 858f902a-21d2-4c13-b3d2-8c4ee62ba67b (dataset), name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 885b9e27-54b2-4713-b093-6d874ca1349c (dataset): DatasetConfig { id: 885b9e27-54b2-4713-b093-6d874ca1349c (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 88906eca-0959-4108-a236-625d08791368 (dataset): DatasetConfig { id: 88906eca-0959-4108-a236-625d08791368 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8c27bc2a-bdd2-4313-97c9-79f3db27f586 (dataset): DatasetConfig { id: 8c27bc2a-bdd2-4313-97c9-79f3db27f586 (dataset), name: DatasetName { pool_name: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8dae09fe-00c8-46ae-b5c3-72be58ce0516 (dataset): DatasetConfig { id: 8dae09fe-00c8-46ae-b5c3-72be58ce0516 (dataset), name: DatasetName { pool_name: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 8e24fefc-519d-4f3a-af94-8e0a70bea3d7 (dataset): DatasetConfig { id: 8e24fefc-519d-4f3a-af94-8e0a70bea3d7 (dataset), name: DatasetName { pool_name: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9247aa19-6e12-4c4c-bf8b-befd2a265431 (dataset): DatasetConfig { id: 9247aa19-6e12-4c4c-bf8b-befd2a265431 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 951a6c80-3a95-48f7-b916-19ac4f14e8e0 (dataset): DatasetConfig { id: 951a6c80-3a95-48f7-b916-19ac4f14e8e0 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: TransientZone { name: "oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 966682e0-68c3-4075-abba-da36520e21df (dataset): DatasetConfig { id: 966682e0-68c3-4075-abba-da36520e21df (dataset), name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9bd3ca48-034a-4de7-aa7f-e05ae7805f3c (dataset): DatasetConfig { id: 9bd3ca48-034a-4de7-aa7f-e05ae7805f3c (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: TransientZone { name: "oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, 9c792e48-71c2-495a-af3a-73fcbf853594 (dataset): DatasetConfig { id: 9c792e48-71c2-495a-af3a-73fcbf853594 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, a2f5fdb5-1433-4caa-b567-19c860c92bb2 (dataset): DatasetConfig { id: a2f5fdb5-1433-4caa-b567-19c860c92bb2 (dataset), name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, afab4ac7-d3ef-4901-a413-20d6cf347bca (dataset): DatasetConfig { id: afab4ac7-d3ef-4901-a413-20d6cf347bca (dataset), name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, b205f29c-82b5-4f47-9ce8-041d18372567 (dataset): DatasetConfig { id: b205f29c-82b5-4f47-9ce8-041d18372567 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Clickhouse }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, babc237b-bd7a-4272-8cb5-b2cabd5aed75 (dataset): DatasetConfig { id: babc237b-bd7a-4272-8cb5-b2cabd5aed75 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, bb12f4bf-3249-411f-9b29-011aa90db3a8 (dataset): DatasetConfig { id: bb12f4bf-3249-411f-9b29-011aa90db3a8 (dataset), name: DatasetName { pool_name: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, bd6b6802-378b-475d-abba-948bfed76657 (dataset): DatasetConfig { id: bd6b6802-378b-475d-abba-948bfed76657 (dataset), name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, c032111e-dfc8-4bc0-a85b-b1481e666919 (dataset): DatasetConfig { id: c032111e-dfc8-4bc0-a85b-b1481e666919 (dataset), name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ce7d683a-d202-43c1-8f0c-a51400165e9a (dataset): DatasetConfig { id: ce7d683a-d202-43c1-8f0c-a51400165e9a (dataset), name: DatasetName { pool_name: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }, kind: TransientZone { name: "oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ce8ab814-584e-4716-922a-431c20480d22 (dataset): DatasetConfig { id: ce8ab814-584e-4716-922a-431c20480d22 (dataset), name: DatasetName { pool_name: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ce974e7d-36ab-4da1-b508-68af2c74f0d4 (dataset): DatasetConfig { id: ce974e7d-36ab-4da1-b508-68af2c74f0d4 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: ExternalDns }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d55c4b03-f011-4fb9-bf0f-5d707bcdf313 (dataset): DatasetConfig { id: d55c4b03-f011-4fb9-bf0f-5d707bcdf313 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, d921c93e-3b4f-4cca-89a8-d2a875645434 (dataset): DatasetConfig { id: d921c93e-3b4f-4cca-89a8-d2a875645434 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: TransientZone { name: "oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e07f87c3-9653-4896-9bed-29768c167e06 (dataset): DatasetConfig { id: e07f87c3-9653-4896-9bed-29768c167e06 (dataset), name: DatasetName { pool_name: ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External }, kind: Debug }, inner: SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } }, e21216a5-4440-4566-92e8-5a940464548d (dataset): DatasetConfig { id: e21216a5-4440-4566-92e8-5a940464548d (dataset), name: DatasetName { pool_name: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e25a0d38-c737-438c-a76b-38da18d40ebc (dataset): DatasetConfig { id: e25a0d38-c737-438c-a76b-38da18d40ebc (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, e64d431c-80f8-459a-8f90-046dbb2eb3ba (dataset): DatasetConfig { id: e64d431c-80f8-459a-8f90-046dbb2eb3ba (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, ed989601-2478-45af-a6fd-32a0620e5978 (dataset): DatasetConfig { id: ed989601-2478-45af-a6fd-32a0620e5978 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f069dbd2-41f5-4bef-8984-2cfbf8404905 (dataset): DatasetConfig { id: f069dbd2-41f5-4bef-8984-2cfbf8404905 (dataset), name: DatasetName { pool_name: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f22a73bf-4f31-4b69-8a91-67ce66648c5c (dataset): DatasetConfig { id: f22a73bf-4f31-4b69-8a91-67ce66648c5c (dataset), name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f4bc7c77-6eaf-4898-b6d9-be3148894069 (dataset): DatasetConfig { id: f4bc7c77-6eaf-4898-b6d9-be3148894069 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f57c966d-590d-4a66-b67b-207616cadf28 (dataset): DatasetConfig { id: f57c966d-590d-4a66-b67b-207616cadf28 (dataset), name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: TransientZone { name: "oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f662aea4-133d-4e18-8adf-7d23948c57df (dataset): DatasetConfig { id: f662aea4-133d-4e18-8adf-7d23948c57df (dataset), name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: TransientZoneRoot }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f736beca-68a0-4673-9319-f7ce56a49190 (dataset): DatasetConfig { id: f736beca-68a0-4673-9319-f7ce56a49190 (dataset), name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Crucible }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }, f7f2c66a-51cf-4098-97b9-a80ecdb5dbb1 (dataset): DatasetConfig { id: f7f2c66a-51cf-4098-97b9-a80ecdb5dbb1 (dataset), name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: TransientZone { name: "oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b" } }, inner: SharedDatasetConfig { compression: Off, quota: None, reservation: None } }} }, tx: ... } file = sled-storage/src/manager.rs:603
4502024-12-13T22:27:27.598ZINFOSledAgent (StorageManager): No ledger in /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-datasets.json file = common/src/ledger.rs:188 request = datasets_ensure
4512024-12-13T22:27:27.598ZINFOSledAgent (StorageManager): No ledger in /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-datasets.json file = common/src/ledger.rs:188 request = datasets_ensure
4522024-12-13T22:27:27.598ZINFOSledAgent (StorageManager): No previously-stored 'requested datasets', creating new ledger file = sled-storage/src/manager.rs:905 request = datasets_ensure
4532024-12-13T22:27:27.598ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4542024-12-13T22:27:27.598ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/external_dns" } file = sled-storage/src/manager.rs:1331
4552024-12-13T22:27:28.097ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4562024-12-13T22:27:28.098ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/fb594314-d689-4df6-8176-e09675ab8f03/crypt/zone/oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2"), full_name: "oxp_fb594314-d689-4df6-8176-e09675ab8f03/crypt/zone/oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2" } file = sled-storage/src/manager.rs:1331
4572024-12-13T22:27:28.452ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/probe_manager.rs:231 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4582024-12-13T22:27:28.734ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4592024-12-13T22:27:28.734ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/35c96663-c574-4d07-a692-3eb8a2a79302/crypt/zone"), full_name: "oxp_35c96663-c574-4d07-a692-3eb8a2a79302/crypt/zone" } file = sled-storage/src/manager.rs:1331
4602024-12-13T22:27:28.946ZWARNSledAgent (producer-registration-task): failed to lookup Nexus IP, will retry delay = 16.811833358s error = "proto error: io error: No route to host (os error 148)" file = oximeter/producer/src/lib.rs:387 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4612024-12-13T22:27:29.109ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4622024-12-13T22:27:29.109ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/b80657b8-dd44-496f-8b60-edeffe6ea0c4/crypt/debug"), full_name: "oxp_b80657b8-dd44-496f-8b60-edeffe6ea0c4/crypt/debug" } file = sled-storage/src/manager.rs:1331
4632024-12-13T22:27:29.389ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4642024-12-13T22:27:29.390ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/internal_dns" } file = sled-storage/src/manager.rs:1331
4652024-12-13T22:27:29.454ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/probe_manager.rs:231 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4662024-12-13T22:27:29.689ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 14907415205677870396 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4672024-12-13T22:27:29.690ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 7111810680181687751 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4682024-12-13T22:27:29.800ZINFOSledAgent (ServiceManager): Ensuring address fd00:1122:3344:101::2 exists - OK file = sled-agent/src/services.rs:4292
4692024-12-13T22:27:29.800ZINFOSledAgent (ServiceManager): Ensuring there is a default route file = sled-agent/src/services.rs:4302 gateway = fd00:1122:3344:101::1
4702024-12-13T22:27:29.806ZINFOSledAgent (ServiceManager): Default route already exists file = sled-agent/src/services.rs:4316 gateway = fd00:1122:3344:101::1
4712024-12-13T22:27:29.806ZINFOSledAgent (ServiceManager): configuring dendrite service file = sled-agent/src/services.rs:4381
4722024-12-13T22:27:29.901ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4732024-12-13T22:27:29.901ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_35c96663-c574-4d07-a692-3eb8a2a79302/crucible" } file = sled-storage/src/manager.rs:1331
4742024-12-13T22:27:29.954ZINFOSledAgent (ServiceManager): refreshed dendrite service with new configuration file = sled-agent/src/services.rs:4447
4752024-12-13T22:27:29.954ZINFOSledAgent (ServiceManager): configuring lldp service file = sled-agent/src/services.rs:4474
4762024-12-13T22:27:29.994ZINFOSledAgent (ServiceManager): refreshed lldpd service with new configuration file = sled-agent/src/services.rs:4487
4772024-12-13T22:27:29.994ZINFOSledAgent (ServiceManager): configuring MGS service file = sled-agent/src/services.rs:4333
4782024-12-13T22:27:30.044ZINFOSledAgent (ServiceManager): refreshed MGS service with new configuration file = sled-agent/src/services.rs:4375
4792024-12-13T22:27:30.044ZINFOSledAgent (ServiceManager): configuring wicketd file = sled-agent/src/services.rs:4454 rack_subnet = fd00:1122:3344::
4802024-12-13T22:27:30.064ZINFOSledAgent (ServiceManager): refreshed wicketd service with new configuration file = sled-agent/src/services.rs:4465
4812024-12-13T22:27:30.064ZINFOSledAgent (ServiceManager): configuring mgd service file = sled-agent/src/services.rs:4507
4822024-12-13T22:27:30.133ZINFOSledAgent (ServiceManager): refreshed mgd service with new configuration file = sled-agent/src/services.rs:4539
4832024-12-13T22:27:30.137ZINFOSledAgent (ServiceManager): configuring mg-ddm service file = sled-agent/src/services.rs:4545
4842024-12-13T22:27:30.229ZINFOSledAgent (ServiceManager): refreshed mg-ddm service with new configuration file = sled-agent/src/services.rs:4586
4852024-12-13T22:27:30.229ZINFOSledAgent (ServiceManager): Determining physical location of our switch zone at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:370
4862024-12-13T22:27:30.265ZINFOSledAgent (ServiceManager): Initializing 1 Uplinks on Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:420
4872024-12-13T22:27:30.266ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/nexus.rs:527 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4882024-12-13T22:27:30.297ZINFOSledAgent (ServiceManager): Building Port Configuration file = sled-agent/src/bootstrap/early_networking.rs:717
4892024-12-13T22:27:30.297ZINFOSledAgent (ServiceManager): Checking dendrite uptime file = sled-agent/src/bootstrap/early_networking.rs:762
4902024-12-13T22:27:30.298ZINFOSledAgent (ServiceManager): Dendrite online file = sled-agent/src/bootstrap/early_networking.rs:765 uptime = 1
4912024-12-13T22:27:30.298ZINFOSledAgent (ServiceManager): Configuring default uplink on switch config = PortSettings {\n links: {\n "0": LinkSettings {\n addrs: [\n 10.151.2.174,\n ],\n params: LinkCreate {\n autoneg: false,\n fec: Some(\n None,\n ),\n kr: false,\n lane: Some(\n LinkId(\n 0,\n ),\n ),\n speed: Speed40G,\n tx_eq: None,\n },\n },\n },\n} file = sled-agent/src/bootstrap/early_networking.rs:444
4922024-12-13T22:27:30.334ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
4932024-12-13T22:27:30.432ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4942024-12-13T22:27:30.432ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/zone/oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2"), full_name: "oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/zone/oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2" } file = sled-storage/src/manager.rs:1331
4952024-12-13T22:27:30.456ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/probe_manager.rs:231 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4962024-12-13T22:27:30.885ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
4972024-12-13T22:27:30.885ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/9e30dd3c-3e72-4b01-8f5f-662db99dc37a/crypt/zone/oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4"), full_name: "oxp_9e30dd3c-3e72-4b01-8f5f-662db99dc37a/crypt/zone/oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4" } file = sled-storage/src/manager.rs:1331
4982024-12-13T22:27:30.890ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17993106840727764337 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
4992024-12-13T22:27:30.890ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17332408252662968763 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
5002024-12-13T22:27:31.338ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5012024-12-13T22:27:31.338ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_5bc1d750-0ca6-4a08-bf34-a01bf34cccef/crucible" } file = sled-storage/src/manager.rs:1331
5022024-12-13T22:27:31.459ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/probe_manager.rs:231 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
5032024-12-13T22:27:31.801ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5042024-12-13T22:27:31.801ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0d689a16-0a28-4b27-af5a-3b4bf99f365f/crypt/debug"), full_name: "oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f/crypt/debug" } file = sled-storage/src/manager.rs:1331
5052024-12-13T22:27:32.138ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5062024-12-13T22:27:32.139ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/ffd405c4-ad68-4401-9d01-3a9d70b7775f/crypt/zone/oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5"), full_name: "oxp_ffd405c4-ad68-4401-9d01-3a9d70b7775f/crypt/zone/oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5" } file = sled-storage/src/manager.rs:1331
5072024-12-13T22:27:32.268ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/nexus.rs:527 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
5082024-12-13T22:27:32.460ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/probe_manager.rs:231 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
5092024-12-13T22:27:32.614ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5102024-12-13T22:27:32.614ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/ffd405c4-ad68-4401-9d01-3a9d70b7775f/crypt/zone"), full_name: "oxp_ffd405c4-ad68-4401-9d01-3a9d70b7775f/crypt/zone" } file = sled-storage/src/manager.rs:1331
5112024-12-13T22:27:32.687ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5122024-12-13T22:27:32.687ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/fb594314-d689-4df6-8176-e09675ab8f03/crypt/zone"), full_name: "oxp_fb594314-d689-4df6-8176-e09675ab8f03/crypt/zone" } file = sled-storage/src/manager.rs:1331
5132024-12-13T22:27:32.759ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5142024-12-13T22:27:32.759ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/9e30dd3c-3e72-4b01-8f5f-662db99dc37a/crypt/zone/oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc"), full_name: "oxp_9e30dd3c-3e72-4b01-8f5f-662db99dc37a/crypt/zone/oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc" } file = sled-storage/src/manager.rs:1331
5152024-12-13T22:27:32.864ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5162024-12-13T22:27:32.864ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/ffd405c4-ad68-4401-9d01-3a9d70b7775f/crypt/zone/oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371"), full_name: "oxp_ffd405c4-ad68-4401-9d01-3a9d70b7775f/crypt/zone/oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371" } file = sled-storage/src/manager.rs:1331
5172024-12-13T22:27:32.968ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5182024-12-13T22:27:32.968ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/fb594314-d689-4df6-8176-e09675ab8f03/crypt/zone/oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981"), full_name: "oxp_fb594314-d689-4df6-8176-e09675ab8f03/crypt/zone/oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981" } file = sled-storage/src/manager.rs:1331
5192024-12-13T22:27:33.074ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5202024-12-13T22:27:33.074ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8159e296-b0fc-4958-9b6f-9386d6e003fa/crypt/zone/oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722"), full_name: "oxp_8159e296-b0fc-4958-9b6f-9386d6e003fa/crypt/zone/oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722" } file = sled-storage/src/manager.rs:1331
5212024-12-13T22:27:33.177ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5222024-12-13T22:27:33.177ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/5bc1d750-0ca6-4a08-bf34-a01bf34cccef/crypt/debug"), full_name: "oxp_5bc1d750-0ca6-4a08-bf34-a01bf34cccef/crypt/debug" } file = sled-storage/src/manager.rs:1331
5232024-12-13T22:27:33.247ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5242024-12-13T22:27:33.247ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0d689a16-0a28-4b27-af5a-3b4bf99f365f/crypt/zone/oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab"), full_name: "oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f/crypt/zone/oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab" } file = sled-storage/src/manager.rs:1331
5252024-12-13T22:27:33.351ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5262024-12-13T22:27:33.351ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f/crucible" } file = sled-storage/src/manager.rs:1331
5272024-12-13T22:27:33.445ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5282024-12-13T22:27:33.445ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/zone/oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874"), full_name: "oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/zone/oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874" } file = sled-storage/src/manager.rs:1331
5292024-12-13T22:27:33.463ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/probe_manager.rs:231 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
5302024-12-13T22:27:33.567ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5312024-12-13T22:27:33.567ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/35c96663-c574-4d07-a692-3eb8a2a79302/crypt/zone/oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919"), full_name: "oxp_35c96663-c574-4d07-a692-3eb8a2a79302/crypt/zone/oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919" } file = sled-storage/src/manager.rs:1331
5322024-12-13T22:27:33.669ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5332024-12-13T22:27:33.669ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8159e296-b0fc-4958-9b6f-9386d6e003fa/crypt/zone/oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c"), full_name: "oxp_8159e296-b0fc-4958-9b6f-9386d6e003fa/crypt/zone/oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c" } file = sled-storage/src/manager.rs:1331
5342024-12-13T22:27:33.778ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5352024-12-13T22:27:33.778ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0d689a16-0a28-4b27-af5a-3b4bf99f365f/crypt/zone/oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a"), full_name: "oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f/crypt/zone/oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a" } file = sled-storage/src/manager.rs:1331
5362024-12-13T22:27:33.883ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5372024-12-13T22:27:33.883ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8159e296-b0fc-4958-9b6f-9386d6e003fa/crypt/zone"), full_name: "oxp_8159e296-b0fc-4958-9b6f-9386d6e003fa/crypt/zone" } file = sled-storage/src/manager.rs:1331
5382024-12-13T22:27:33.956ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5392024-12-13T22:27:33.956ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_9e30dd3c-3e72-4b01-8f5f-662db99dc37a/crucible" } file = sled-storage/src/manager.rs:1331
5402024-12-13T22:27:34.051ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5412024-12-13T22:27:34.052ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/ffd405c4-ad68-4401-9d01-3a9d70b7775f/crypt/debug"), full_name: "oxp_ffd405c4-ad68-4401-9d01-3a9d70b7775f/crypt/debug" } file = sled-storage/src/manager.rs:1331
5422024-12-13T22:27:34.126ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5432024-12-13T22:27:34.126ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/debug"), full_name: "oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/debug" } file = sled-storage/src/manager.rs:1331
5442024-12-13T22:27:34.197ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5452024-12-13T22:27:34.197ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_fb594314-d689-4df6-8176-e09675ab8f03/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
5462024-12-13T22:27:34.267ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/nexus.rs:527 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
5472024-12-13T22:27:34.293ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5482024-12-13T22:27:34.293ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/5bc1d750-0ca6-4a08-bf34-a01bf34cccef/crypt/zone"), full_name: "oxp_5bc1d750-0ca6-4a08-bf34-a01bf34cccef/crypt/zone" } file = sled-storage/src/manager.rs:1331
5492024-12-13T22:27:34.364ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5502024-12-13T22:27:34.364ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
5512024-12-13T22:27:34.457ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5522024-12-13T22:27:34.457ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/9e30dd3c-3e72-4b01-8f5f-662db99dc37a/crypt/zone/oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927"), full_name: "oxp_9e30dd3c-3e72-4b01-8f5f-662db99dc37a/crypt/zone/oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927" } file = sled-storage/src/manager.rs:1331
5532024-12-13T22:27:34.468ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/probe_manager.rs:231 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
5542024-12-13T22:27:34.559ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5552024-12-13T22:27:34.559ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/zone/oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80"), full_name: "oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/zone/oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80" } file = sled-storage/src/manager.rs:1331
5562024-12-13T22:27:34.662ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5572024-12-13T22:27:34.662ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f/crypt/internal_dns" } file = sled-storage/src/manager.rs:1331
5582024-12-13T22:27:34.755ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5592024-12-13T22:27:34.755ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/b80657b8-dd44-496f-8b60-edeffe6ea0c4/crypt/zone/oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869"), full_name: "oxp_b80657b8-dd44-496f-8b60-edeffe6ea0c4/crypt/zone/oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869" } file = sled-storage/src/manager.rs:1331
5602024-12-13T22:27:34.860ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5612024-12-13T22:27:34.860ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_ffd405c4-ad68-4401-9d01-3a9d70b7775f/crucible" } file = sled-storage/src/manager.rs:1331
5622024-12-13T22:27:34.954ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5632024-12-13T22:27:34.954ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/5bc1d750-0ca6-4a08-bf34-a01bf34cccef/crypt/zone/oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c"), full_name: "oxp_5bc1d750-0ca6-4a08-bf34-a01bf34cccef/crypt/zone/oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c" } file = sled-storage/src/manager.rs:1331
5642024-12-13T22:27:35.061ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5652024-12-13T22:27:35.061ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0d689a16-0a28-4b27-af5a-3b4bf99f365f/crypt/zone/oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a"), full_name: "oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f/crypt/zone/oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a" } file = sled-storage/src/manager.rs:1331
5662024-12-13T22:27:35.165ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5672024-12-13T22:27:35.165ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8159e296-b0fc-4958-9b6f-9386d6e003fa/crypt/zone/oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3"), full_name: "oxp_8159e296-b0fc-4958-9b6f-9386d6e003fa/crypt/zone/oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3" } file = sled-storage/src/manager.rs:1331
5682024-12-13T22:27:35.270ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5692024-12-13T22:27:35.270ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_9e30dd3c-3e72-4b01-8f5f-662db99dc37a/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
5702024-12-13T22:27:35.367ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5712024-12-13T22:27:35.367ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/zone/oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae"), full_name: "oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/zone/oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae" } file = sled-storage/src/manager.rs:1331
5722024-12-13T22:27:35.471ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/probe_manager.rs:231 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
5732024-12-13T22:27:35.472ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5742024-12-13T22:27:35.472ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/8159e296-b0fc-4958-9b6f-9386d6e003fa/crypt/debug"), full_name: "oxp_8159e296-b0fc-4958-9b6f-9386d6e003fa/crypt/debug" } file = sled-storage/src/manager.rs:1331
5752024-12-13T22:27:35.545ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5762024-12-13T22:27:35.545ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/fb594314-d689-4df6-8176-e09675ab8f03/crypt/debug"), full_name: "oxp_fb594314-d689-4df6-8176-e09675ab8f03/crypt/debug" } file = sled-storage/src/manager.rs:1331
5772024-12-13T22:27:35.617ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5782024-12-13T22:27:35.617ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/fb594314-d689-4df6-8176-e09675ab8f03/crypt/zone/oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5"), full_name: "oxp_fb594314-d689-4df6-8176-e09675ab8f03/crypt/zone/oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5" } file = sled-storage/src/manager.rs:1331
5792024-12-13T22:27:35.719ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5802024-12-13T22:27:35.719ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/clickhouse" } file = sled-storage/src/manager.rs:1331
5812024-12-13T22:27:35.812ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5822024-12-13T22:27:35.812ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0d689a16-0a28-4b27-af5a-3b4bf99f365f/crypt/zone"), full_name: "oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f/crypt/zone" } file = sled-storage/src/manager.rs:1331
5832024-12-13T22:27:35.883ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5842024-12-13T22:27:35.883ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/ffd405c4-ad68-4401-9d01-3a9d70b7775f/crypt/zone/oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934"), full_name: "oxp_ffd405c4-ad68-4401-9d01-3a9d70b7775f/crypt/zone/oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934" } file = sled-storage/src/manager.rs:1331
5852024-12-13T22:27:35.992ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5862024-12-13T22:27:35.992ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/9e30dd3c-3e72-4b01-8f5f-662db99dc37a/crypt/debug"), full_name: "oxp_9e30dd3c-3e72-4b01-8f5f-662db99dc37a/crypt/debug" } file = sled-storage/src/manager.rs:1331
5872024-12-13T22:27:36.065ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5882024-12-13T22:27:36.065ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_9e30dd3c-3e72-4b01-8f5f-662db99dc37a/crypt/internal_dns" } file = sled-storage/src/manager.rs:1331
5892024-12-13T22:27:36.162ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5902024-12-13T22:27:36.162ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/5bc1d750-0ca6-4a08-bf34-a01bf34cccef/crypt/zone/oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93"), full_name: "oxp_5bc1d750-0ca6-4a08-bf34-a01bf34cccef/crypt/zone/oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93" } file = sled-storage/src/manager.rs:1331
5912024-12-13T22:27:36.266ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5922024-12-13T22:27:36.266ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/b80657b8-dd44-496f-8b60-edeffe6ea0c4/crypt/zone/oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4"), full_name: "oxp_b80657b8-dd44-496f-8b60-edeffe6ea0c4/crypt/zone/oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4" } file = sled-storage/src/manager.rs:1331
5932024-12-13T22:27:36.270ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/nexus.rs:527 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
5942024-12-13T22:27:36.371ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5952024-12-13T22:27:36.371ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f/crypt/external_dns" } file = sled-storage/src/manager.rs:1331
5962024-12-13T22:27:36.465ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
5972024-12-13T22:27:36.465ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5/crucible" } file = sled-storage/src/manager.rs:1331
5982024-12-13T22:27:36.476ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/probe_manager.rs:231 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
5992024-12-13T22:27:36.560ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6002024-12-13T22:27:36.560ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/zone/oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9"), full_name: "oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/zone/oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9" } file = sled-storage/src/manager.rs:1331
6012024-12-13T22:27:36.663ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6022024-12-13T22:27:36.663ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: GzipN { level: GzipLevel(9) }, quota: Some(ByteCount(107374182400)), reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/35c96663-c574-4d07-a692-3eb8a2a79302/crypt/debug"), full_name: "oxp_35c96663-c574-4d07-a692-3eb8a2a79302/crypt/debug" } file = sled-storage/src/manager.rs:1331
6032024-12-13T22:27:36.736ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6042024-12-13T22:27:36.736ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_b80657b8-dd44-496f-8b60-edeffe6ea0c4/crucible" } file = sled-storage/src/manager.rs:1331
6052024-12-13T22:27:36.826ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6062024-12-13T22:27:36.826ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/zone"), full_name: "oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/zone" } file = sled-storage/src/manager.rs:1331
6072024-12-13T22:27:36.899ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6082024-12-13T22:27:36.899ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
6092024-12-13T22:27:36.998ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6102024-12-13T22:27:36.998ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/zone/oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c"), full_name: "oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/zone/oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c" } file = sled-storage/src/manager.rs:1331
6112024-12-13T22:27:37.101ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6122024-12-13T22:27:37.101ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/b80657b8-dd44-496f-8b60-edeffe6ea0c4/crypt/zone"), full_name: "oxp_b80657b8-dd44-496f-8b60-edeffe6ea0c4/crypt/zone" } file = sled-storage/src/manager.rs:1331
6132024-12-13T22:27:37.175ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6142024-12-13T22:27:37.175ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_fb594314-d689-4df6-8176-e09675ab8f03/crucible" } file = sled-storage/src/manager.rs:1331
6152024-12-13T22:27:37.271ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6162024-12-13T22:27:37.271ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_8159e296-b0fc-4958-9b6f-9386d6e003fa/crypt/cockroachdb" } file = sled-storage/src/manager.rs:1331
6172024-12-13T22:27:37.367ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6182024-12-13T22:27:37.367ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/zone/oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0"), full_name: "oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/zone/oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0" } file = sled-storage/src/manager.rs:1331
6192024-12-13T22:27:37.470ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6202024-12-13T22:27:37.470ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/9e30dd3c-3e72-4b01-8f5f-662db99dc37a/crypt/zone"), full_name: "oxp_9e30dd3c-3e72-4b01-8f5f-662db99dc37a/crypt/zone" } file = sled-storage/src/manager.rs:1331
6212024-12-13T22:27:37.478ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/probe_manager.rs:231 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
6222024-12-13T22:27:37.540ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6232024-12-13T22:27:37.540ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: true, mountpoint: Path("/data"), full_name: "oxp_8159e296-b0fc-4958-9b6f-9386d6e003fa/crucible" } file = sled-storage/src/manager.rs:1331
6242024-12-13T22:27:37.634ZINFOSledAgent (StorageManager): Ensuring dataset file = sled-storage/src/manager.rs:949 request = datasets_ensure
6252024-12-13T22:27:37.634ZINFOSledAgent (StorageManager): ensure_dataset config = SharedDatasetConfig { compression: Off, quota: None, reservation: None } details = DatasetCreationDetails { zoned: false, mountpoint: Path("/pool/ext/0d689a16-0a28-4b27-af5a-3b4bf99f365f/crypt/zone/oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b"), full_name: "oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f/crypt/zone/oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b" } file = sled-storage/src/manager.rs:1331
6262024-12-13T22:27:37.736ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/.omicron-datasets.json.tmp file = common/src/ledger.rs:199 request = datasets_ensure
6272024-12-13T22:27:37.736ZINFOSledAgent (StorageManager): Writing ledger to /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/.omicron-datasets.json.tmp file = common/src/ledger.rs:199 request = datasets_ensure
6282024-12-13T22:27:37.736ZINFOSledAgent (StorageManager): Received DetectedRawDisksChanged { raw_disks: {}, tx: ... } file = sled-storage/src/manager.rs:603
6292024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
6302024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-0" }
6312024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
6322024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-M2", serial: "synthetic-serial-1" }
6332024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
6342024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-0" }
6352024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
6362024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-1" }
6372024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
6382024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-2" }
6392024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
6402024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-3" }
6412024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
6422024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-4" }
6432024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
6442024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-5" }
6452024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
6462024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-6" }
6472024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
6482024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-7" }
6492024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Removing disk file = sled-storage/src/resources.rs:529 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
6502024-12-13T22:27:37.736ZINFOSledAgent (StorageResources): Not removing synthetic disk file = sled-storage/src/resources.rs:550 identity = DiskIdentity { vendor: "synthetic-vendor", model: "synthetic-model-U2", serial: "synthetic-serial-8" }
6512024-12-13T22:27:37.736ZINFOSledAgent: datasets ensure: Updated storage file = sled-agent/src/sled_agent.rs:861 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
6522024-12-13T22:27:37.737ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 10139218 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:63873 req_id = 1afe42f2-c0fe-4472-9b23-ba0d2c82aaab response_code = 200 uri = /datasets
6532024-12-13T22:27:37.768ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(2), zones: [OmicronZoneConfig { id: 395a4d9e-1527-4351-8190-c1cd55768e7c (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
6542024-12-13T22:27:37.768ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:39431
6552024-12-13T22:27:37.769ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
6562024-12-13T22:27:37.769ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }
6572024-12-13T22:27:37.803ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
6582024-12-13T22:27:37.804ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }
6592024-12-13T22:27:37.835ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
6602024-12-13T22:27:37.835ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }
6612024-12-13T22:27:37.869ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6622024-12-13T22:27:37.869ZINFOSledAgent (ServiceManager): No ledger in /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:188
6632024-12-13T22:27:37.869ZINFOSledAgent (ServiceManager): No ledger in /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:188
6642024-12-13T22:27:37.869ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6652024-12-13T22:27:37.916ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6662024-12-13T22:27:37.916ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6672024-12-13T22:27:37.916ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
6682024-12-13T22:27:37.972ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc file = illumos-utils/src/zone.rs:325
6692024-12-13T22:27:37.976ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c file = illumos-utils/src/zone.rs:325
6702024-12-13T22:27:37.980ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b file = illumos-utils/src/zone.rs:325
6712024-12-13T22:27:37.992ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc file = illumos-utils/src/zone.rs:363
6722024-12-13T22:27:37.997ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c file = illumos-utils/src/zone.rs:363
6732024-12-13T22:27:38.001ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b file = illumos-utils/src/zone.rs:363
6742024-12-13T22:27:38.267ZWARNSledAgent (NexusNotifierTask): Received Error from Nexus for Get request: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/sled-agents/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/nexus.rs:527 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
6752024-12-13T22:27:38.480ZERROSledAgent (ProbeManager): get target probe state: Communication Error: error sending request for url (http://_nexus._tcp.control-plane.oxide.internal:12221/probes/35eb16d3-adec-4806-a15c-ab7a15d18765) file = sled-agent/src/probe_manager.rs:231 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
6762024-12-13T22:27:39.610ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService2'/> <propval type="astring" name="gateway" value='fd00:1122:3344:1::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:1::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:1::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:1::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6772024-12-13T22:27:39.611ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:1::, width: 64 }
6782024-12-13T22:27:39.691ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 14907415205677870396 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
6792024-12-13T22:27:39.692ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 7111810680181687751 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
6802024-12-13T22:27:40.090ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService1'/> <propval type="astring" name="gateway" value='fd00:1122:3344:3::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:3::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:3::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:3::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6812024-12-13T22:27:40.090ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c
6822024-12-13T22:27:40.576ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:3::, width: 64 }
6832024-12-13T22:27:40.576ZINFOSledAgent (ServiceManager): Profile for oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService3'/> <propval type="astring" name="gateway" value='fd00:1122:3344:2::2'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:2::1'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/internal_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:2::1]:5353'/> <propval type="astring" name="dns_address" value='[fd00:1122:3344:2::1]:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
6842024-12-13T22:27:40.576ZINFOSledAgent (BootstrapAgentStartup): Sending prefix to ddmd for advertisement DdmAdminClient = [::1]:8000 file = clients/ddm-admin-client/src/lib.rs:86 prefix = Ipv6Net { addr: fd00:1122:3344:2::, width: 64 }
6852024-12-13T22:27:40.576ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc
6862024-12-13T22:27:40.592ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c") failed: Property not found. retry in 62.625688ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c
6872024-12-13T22:27:40.592ZINFOSledAgent (ServiceManager): Booting oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b
6882024-12-13T22:27:40.664ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c") failed: Property not found. retry in 133.113573ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c
6892024-12-13T22:27:40.706ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc") failed: Property not found. retry in 37.27118ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc
6902024-12-13T22:27:40.731ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b") failed: Property not found. retry in 43.994888ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b
6912024-12-13T22:27:40.751ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc") failed: Property not found. retry in 94.41113ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc
6922024-12-13T22:27:40.793ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b") failed: Property not found. retry in 117.873551ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b
6932024-12-13T22:27:40.868ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c") failed: Property not found. retry in 263.456467ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c
6942024-12-13T22:27:40.982ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17993106840727764337 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
6952024-12-13T22:27:40.983ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17332408252662968763 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
6962024-12-13T22:27:40.992ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc") failed: Property not found. retry in 182.184169ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc
6972024-12-13T22:27:41.009ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b") failed: Property not found. retry in 102.198199ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b
6982024-12-13T22:27:41.134ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b") failed: Property not found. retry in 588.422459ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b
6992024-12-13T22:27:41.152ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c") failed: Property not found. retry in 388.55581ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c
7002024-12-13T22:27:41.188ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc") failed: Property not found. retry in 245.00953ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc
7012024-12-13T22:27:41.443ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc") failed: Property not found. retry in 1.053510053s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc
7022024-12-13T22:27:41.549ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c") failed: Property not found. retry in 823.67636ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c
7032024-12-13T22:27:41.732ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b") failed: Property not found. retry in 783.746682ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b
7042024-12-13T22:27:42.383ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c") failed: Property not found. retry in 571.785937ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c
7052024-12-13T22:27:42.509ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc") failed: Property not found. retry in 1.487027412s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc
7062024-12-13T22:27:42.528ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b") failed: Property not found. retry in 949.588884ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b
7072024-12-13T22:27:42.964ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c") failed: Property not found. retry in 671.417873ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c
7082024-12-13T22:27:43.486ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b") failed: Property not found. retry in 831.692671ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b
7092024-12-13T22:27:43.644ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c") failed: Property not found. retry in 1.157805139s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c
7102024-12-13T22:27:44.004ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc") failed: Property not found. retry in 1.193207582s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc
7112024-12-13T22:27:44.326ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b") failed: Property not found. retry in 1.109965728s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b
7122024-12-13T22:27:44.811ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c") failed: Property not found. retry in 738.612704ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c
7132024-12-13T22:27:45.208ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc") failed: Property not found. retry in 898.86487ms file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc
7142024-12-13T22:27:45.447ZWARNSledAgent (ServiceManager): wait for service Some("oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b") failed: Property not found. retry in 1.4404045s file = illumos-utils/src/svc.rs:36 zone = oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b
7152024-12-13T22:27:46.907ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_internal_dns_b14af4bc-de06-49b6-a6f5-2ef07a34b0dc
7162024-12-13T22:27:46.907ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_internal_dns_395a4d9e-1527-4351-8190-c1cd55768e7c
7172024-12-13T22:27:46.907ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_internal_dns_2bddd59f-8fa2-4b3b-ae1a-b309cf00641b
7182024-12-13T22:27:46.907ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7192024-12-13T22:27:46.908ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7202024-12-13T22:27:46.908ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9139374 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:39431 req_id = 6beb1a5e-05db-47bc-80d1-d7e0670e5dbf response_code = 204 uri = /omicron-zones
7212024-12-13T22:27:46.910ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:1::1]:5353 file = sled-agent/src/rack_setup/service.rs:658
7222024-12-13T22:27:46.958ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:2::1]:5353 file = sled-agent/src/rack_setup/service.rs:658
7232024-12-13T22:27:47.005ZINFOSledAgent (RSS): Configuring DNS server dns_config_addr = [fd00:1122:3344:3::1]:5353 file = sled-agent/src/rack_setup/service.rs:658
7242024-12-13T22:27:47.052ZINFOSledAgent (RSS): Configured all DNS servers file = sled-agent/src/rack_setup/service.rs:694
7252024-12-13T22:27:47.052ZINFOSledAgent (RSS): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
7262024-12-13T22:27:47.084ZINFOSledAgent (RSS): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
7272024-12-13T22:27:47.085ZINFOSledAgent (RSS): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
7282024-12-13T22:27:47.085ZINFOSledAgent (RSS): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
7292024-12-13T22:27:47.114ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(3), zones: [OmicronZoneConfig { id: 395a4d9e-1527-4351-8190-c1cd55768e7c (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 (service), filesystem_pool: Some(ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 89fc6686-a783-492f-b4f9-cc69952dca5b, kind: Service { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 }, name: Name("ntp-1f4a9057-ec2f-4d3c-b6c8-1716587b7371"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 109])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
7302024-12-13T22:27:47.114ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:39293
7312024-12-13T22:27:47.115ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
7322024-12-13T22:27:47.115ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }
7332024-12-13T22:27:47.154ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
7342024-12-13T22:27:47.154ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }
7352024-12-13T22:27:47.194ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
7362024-12-13T22:27:47.194ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }
7372024-12-13T22:27:47.229ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
7382024-12-13T22:27:47.229ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:177
7392024-12-13T22:27:47.229ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:177
7402024-12-13T22:27:47.230ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
7412024-12-13T22:27:47.240ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
7422024-12-13T22:27:47.273ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
7432024-12-13T22:27:47.273ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
7442024-12-13T22:27:47.273ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
7452024-12-13T22:27:47.303ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte0", ip: 172.30.3.5, mac: MacAddr6([168, 64, 37, 255, 216, 109]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.3.1 } }) }
7462024-12-13T22:27:47.303ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1263 zone_type = boundary_ntp
7472024-12-13T22:27:47.303ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:83
7482024-12-13T22:27:47.303ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:90
7492024-12-13T22:27:47.303ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.169 file = clients/dpd-client/src/lib.rs:118
7502024-12-13T22:27:47.304ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
7512024-12-13T22:27:47.328ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371 file = illumos-utils/src/zone.rs:325
7522024-12-13T22:27:47.346ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371 file = illumos-utils/src/zone.rs:363
7532024-12-13T22:27:47.976ZINFOSledAgent (ServiceManager): Profile for oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService4'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/chrony-setup"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="allow" value='fd00:1122:3344:100::/56'/> <propval type="boolean" name="boundary" value='true'/> <propval type="astring" name="boundary_pool" value='boundary-ntp.control-plane.oxide.internal'/> <propval type="astring" name="server" value='0.pool.ntp.org'/> </property_group> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="1.1.1.1" /> <value_node value="9.9.9.9" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/ntp"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte0'/> <propval type="astring" name="gateway" value='172.30.3.1'/> <propval type="astring" name="ip" value='172.30.3.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
7542024-12-13T22:27:47.976ZINFOSledAgent (ServiceManager): Booting oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371
7552024-12-13T22:27:48.116ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371") failed: Property not found. retry in 37.805827ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371
7562024-12-13T22:27:48.162ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371") failed: Property not found. retry in 80.873049ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371
7572024-12-13T22:27:48.255ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371") failed: Property not found. retry in 161.656348ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371
7582024-12-13T22:27:48.428ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371") failed: Property not found. retry in 244.733705ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371
7592024-12-13T22:27:48.682ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371") failed: Property not found. retry in 478.551088ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371
7602024-12-13T22:27:49.170ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371") failed: Property not found. retry in 636.090396ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371
7612024-12-13T22:27:49.694ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 14907415205677870396 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
7622024-12-13T22:27:49.694ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 7111810680181687751 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
7632024-12-13T22:27:49.815ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371") failed: Property not found. retry in 737.196262ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371
7642024-12-13T22:27:50.562ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371") failed: Property not found. retry in 688.082719ms file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371
7652024-12-13T22:27:50.983ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17993106840727764337 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
7662024-12-13T22:27:50.983ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17332408252662968763 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
7672024-12-13T22:27:51.258ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371") failed: Property not found. retry in 1.388507057s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371
7682024-12-13T22:27:52.656ZWARNSledAgent (ServiceManager): wait for service Some("oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371") failed: Property not found. retry in 1.066520051s file = illumos-utils/src/svc.rs:36 zone = oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371
7692024-12-13T22:27:53.743ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371
7702024-12-13T22:27:53.743ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7712024-12-13T22:27:53.743ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
7722024-12-13T22:27:53.744ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6629121 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:39293 req_id = dcce1945-37a2-4fc8-95a9-522bc1b1f9c6 response_code = 204 uri = /omicron-zones
7732024-12-13T22:27:53.746ZINFOSledAgent (RSS): Waiting for rack time synchronization file = sled-agent/src/rack_setup/service.rs:735
7742024-12-13T22:27:53.777ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
7752024-12-13T22:27:53.777ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:53651
7762024-12-13T22:27:53.786ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3900
7772024-12-13T22:27:53.786ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 8962 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:53651 req_id = c6c4a016-0233-4cf1-b0f2-2dc2d73ca21e response_code = 500 uri = /timesync
7782024-12-13T22:27:53.788ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
7792024-12-13T22:27:53.874ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
7802024-12-13T22:27:53.874ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:48627
7812024-12-13T22:27:55.576ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 6273088363219224010 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
7822024-12-13T22:27:55.883ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3900
7832024-12-13T22:27:55.883ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 2008812 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:48627 req_id = 04ead620-6d47-4021-a166-251115560b9a response_code = 500 uri = /timesync
7842024-12-13T22:27:55.885ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
7852024-12-13T22:27:56.080ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
7862024-12-13T22:27:56.080ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:63467
7872024-12-13T22:27:56.089ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3900
7882024-12-13T22:27:56.089ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 8534 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:63467 req_id = 56855e85-b7db-4c7a-bebb-d2ffd6cf564f response_code = 500 uri = /timesync
7892024-12-13T22:27:56.092ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
7902024-12-13T22:27:56.131ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 5912078162058704635 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
7912024-12-13T22:27:56.297ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
7922024-12-13T22:27:56.297ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:42770
7932024-12-13T22:27:56.305ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3900
7942024-12-13T22:27:56.305ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 7746 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:42770 req_id = 2434149f-0a9c-4fa3-9f4d-b5bcd9e120b2 response_code = 500 uri = /timesync
7952024-12-13T22:27:56.308ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
7962024-12-13T22:27:56.591ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
7972024-12-13T22:27:56.592ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:40114
7982024-12-13T22:27:56.911ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 6402808983019246285 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
7992024-12-13T22:27:57.602ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3900
8002024-12-13T22:27:57.602ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 1010460 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:40114 req_id = b9b8d5b9-dd9d-4da7-b869-818fb85b308e response_code = 500 uri = /timesync
8012024-12-13T22:27:57.604ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8022024-12-13T22:27:57.869ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8032024-12-13T22:27:57.869ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:61085
8042024-12-13T22:27:57.878ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3900
8052024-12-13T22:27:57.878ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 9211 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:61085 req_id = 5bf862f1-8749-4f4b-ae13-cfd9e104af09 response_code = 500 uri = /timesync
8062024-12-13T22:27:57.882ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8072024-12-13T22:27:58.166ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8082024-12-13T22:27:58.166ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:53440
8092024-12-13T22:27:58.175ZERROSledAgent (ServiceManager): chronyc command failed: Error running command in zone 'oxz_ntp_1f4a9057-ec2f-4d3c-b6c8-1716587b7371': Command [/usr/bin/chronyc -c tracking] executed and failed with status: exit status: 1 stdout: 506 Cannot talk to daemon stderr: file = sled-agent/src/services.rs:3900
8102024-12-13T22:27:58.175ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = NTP zone not ready file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 8806 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:53440 req_id = 31c75221-6945-472a-ac7e-ed48aa953d2c response_code = 500 uri = /timesync
8112024-12-13T22:27:58.178ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8122024-12-13T22:27:58.551ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8132024-12-13T22:27:58.551ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:37872
8142024-12-13T22:27:59.695ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 14907415205677870396 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
8152024-12-13T22:27:59.696ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 7111810680181687751 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
8162024-12-13T22:28:00.007ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:50315
8172024-12-13T22:28:00.007ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 138 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:50315 req_id = 8702925e-c708-456f-9630-0fb93e427bd1 response_code = 200 uri = /baseboard
8182024-12-13T22:28:00.282ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2009457 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:37872 req_id = f3016ab0-f5c4-44bb-a9ad-0c4067b65f65 response_code = 200 uri = /timesync
8192024-12-13T22:28:00.285ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.027760336 } file = sled-agent/src/rack_setup/service.rs:743
8202024-12-13T22:28:00.285ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8212024-12-13T22:28:00.708ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17993106840727764337 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
8222024-12-13T22:28:00.708ZERROSledAgent (kstat-sampler-worker): failed to sample kstat target, requeueing error = NoSuchKstat file = oximeter/instruments/src/kstat/sampler.rs:496 id = 17332408252662968763 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
8232024-12-13T22:28:00.810ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8242024-12-13T22:28:00.810ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64335
8252024-12-13T22:28:00.819ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9364 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:64335 req_id = 5f1d35bf-5e3f-47c1-8c25-76699ac0407e response_code = 200 uri = /timesync
8262024-12-13T22:28:00.821ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.026300224 } file = sled-agent/src/rack_setup/service.rs:743
8272024-12-13T22:28:00.821ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8282024-12-13T22:28:01.249ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8292024-12-13T22:28:01.249ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:54470
8302024-12-13T22:28:01.258ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8716 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:54470 req_id = 9b64541d-ad78-43d0-ae15-6d011b098217 response_code = 200 uri = /timesync
8312024-12-13T22:28:01.261ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.030073017 } file = sled-agent/src/rack_setup/service.rs:743
8322024-12-13T22:28:01.261ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8332024-12-13T22:28:02.025ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8342024-12-13T22:28:02.025ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:59580
8352024-12-13T22:28:02.033ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8264 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:59580 req_id = fe5c889a-9bc9-4ee0-b3c4-302b4204a903 response_code = 200 uri = /timesync
8362024-12-13T22:28:02.035ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: false, ref_id: 0, ip_addr: ::, stratum: 0, ref_time: 0.0, correction: -0.027967388 } file = sled-agent/src/rack_setup/service.rs:743
8372024-12-13T22:28:02.035ZWARNSledAgent (RSS): Time is not yet synchronized error = "Time is synchronized on 0/1 sleds" file = sled-agent/src/rack_setup/service.rs:765
8382024-12-13T22:28:02.393ZINFOSledAgent (RSS): Checking time synchronization for [fd00:1122:3344:101::1]:12345... file = sled-agent/src/rack_setup/service.rs:715
8392024-12-13T22:28:02.393ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:60034
8402024-12-13T22:28:02.414ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 19962 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::1]:60034 req_id = cdea215c-4843-4d99-8f4a-46f876933494 response_code = 200 uri = /timesync
8412024-12-13T22:28:02.417ZINFOSledAgent (RSS): Timesync for [fd00:1122:3344:101::1]:12345 TimeSync { sync: true, ref_id: 2429174486, ip_addr: 144.202.66.214, stratum: 6, ref_time: 1734128882.1007316, correction: -0.033054091 } file = sled-agent/src/rack_setup/service.rs:743
8422024-12-13T22:28:02.417ZINFOSledAgent (RSS): Finished setting up Internal DNS and NTP file = sled-agent/src/rack_setup/service.rs:1392
8432024-12-13T22:28:02.446ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 395a4d9e-1527-4351-8190-c1cd55768e7c (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 (service), filesystem_pool: Some(ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 89fc6686-a783-492f-b4f9-cc69952dca5b, kind: Service { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 }, name: Name("ntp-1f4a9057-ec2f-4d3c-b6c8-1716587b7371"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 109])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: d623e625-b63d-4f90-882e-385f19de4874 (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } } } }, OmicronZoneConfig { id: ab512f01-b585-4697-95ff-95548582574a (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } } } }, OmicronZoneConfig { id: 1af22521-3df5-4f6c-b052-3f1bff202927 (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } } } }, OmicronZoneConfig { id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3 (service), filesystem_pool: Some(ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 6bb86fdc-adfa-4396-af16-3194717465e2 (service), filesystem_pool: Some(ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
8442024-12-13T22:28:02.446ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:49397
8452024-12-13T22:28:02.447ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8462024-12-13T22:28:02.447ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8472024-12-13T22:28:02.481ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8482024-12-13T22:28:02.482ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }
8492024-12-13T22:28:02.512ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8502024-12-13T22:28:02.512ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }
8512024-12-13T22:28:02.541ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8522024-12-13T22:28:02.541ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8532024-12-13T22:28:02.571ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8542024-12-13T22:28:02.571ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }
8552024-12-13T22:28:02.601ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8562024-12-13T22:28:02.601ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }
8572024-12-13T22:28:02.634ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8582024-12-13T22:28:02.634ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
8592024-12-13T22:28:02.665ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8602024-12-13T22:28:02.665ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8612024-12-13T22:28:02.695ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8622024-12-13T22:28:02.695ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:177
8632024-12-13T22:28:02.696ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:177
8642024-12-13T22:28:02.696ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8652024-12-13T22:28:02.729ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
8662024-12-13T22:28:02.729ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
8672024-12-13T22:28:02.729ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
8682024-12-13T22:28:02.729ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
8692024-12-13T22:28:02.729ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
8702024-12-13T22:28:02.729ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874", TimeNotSynchronized), ("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a", TimeNotSynchronized), ("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927", TimeNotSynchronized), ("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3", TimeNotSynchronized), ("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 283428 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:49397 req_id = 38348c54-c25b-4d14-8c39-db4aed7a3585 response_code = 500 uri = /omicron-zones
8712024-12-13T22:28:02.729ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "38348c54-c25b-4d14-8c39-db4aed7a3585", "content-length": "124", "date": "Fri, 13 Dec 2024 22:28:02 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "38348c54-c25b-4d14-8c39-db4aed7a3585" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 103.72304ms sled_address = [fd00:1122:3344:101::1]:12345
8722024-12-13T22:28:02.834ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 395a4d9e-1527-4351-8190-c1cd55768e7c (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 (service), filesystem_pool: Some(ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 89fc6686-a783-492f-b4f9-cc69952dca5b, kind: Service { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 }, name: Name("ntp-1f4a9057-ec2f-4d3c-b6c8-1716587b7371"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 109])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: d623e625-b63d-4f90-882e-385f19de4874 (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } } } }, OmicronZoneConfig { id: ab512f01-b585-4697-95ff-95548582574a (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } } } }, OmicronZoneConfig { id: 1af22521-3df5-4f6c-b052-3f1bff202927 (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } } } }, OmicronZoneConfig { id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3 (service), filesystem_pool: Some(ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 6bb86fdc-adfa-4396-af16-3194717465e2 (service), filesystem_pool: Some(ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
8732024-12-13T22:28:02.835ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8742024-12-13T22:28:02.835ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }
8752024-12-13T22:28:02.867ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8762024-12-13T22:28:02.867ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }
8772024-12-13T22:28:02.898ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
8782024-12-13T22:28:02.898ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }
8792024-12-13T22:28:02.929ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8802024-12-13T22:28:02.929ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8812024-12-13T22:28:02.960ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8822024-12-13T22:28:02.961ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }
8832024-12-13T22:28:02.992ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8842024-12-13T22:28:02.992ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }
8852024-12-13T22:28:03.024ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8862024-12-13T22:28:03.024ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
8872024-12-13T22:28:03.056ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
8882024-12-13T22:28:03.056ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }
8892024-12-13T22:28:03.087ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8902024-12-13T22:28:03.087ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:177
8912024-12-13T22:28:03.087ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:177
8922024-12-13T22:28:03.088ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
8932024-12-13T22:28:03.120ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
8942024-12-13T22:28:03.120ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
8952024-12-13T22:28:03.120ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
8962024-12-13T22:28:03.120ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
8972024-12-13T22:28:03.120ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
8982024-12-13T22:28:03.120ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927", TimeNotSynchronized), ("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874", TimeNotSynchronized), ("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a", TimeNotSynchronized), ("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2", TimeNotSynchronized), ("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 286713 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:49397 req_id = 633f3856-eaaa-4b92-876c-d1bebf555148 response_code = 500 uri = /omicron-zones
8992024-12-13T22:28:03.120ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "633f3856-eaaa-4b92-876c-d1bebf555148", "content-length": "124", "date": "Fri, 13 Dec 2024 22:28:02 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "633f3856-eaaa-4b92-876c-d1bebf555148" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 127.599919ms sled_address = [fd00:1122:3344:101::1]:12345
9002024-12-13T22:28:03.248ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 395a4d9e-1527-4351-8190-c1cd55768e7c (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 (service), filesystem_pool: Some(ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 89fc6686-a783-492f-b4f9-cc69952dca5b, kind: Service { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 }, name: Name("ntp-1f4a9057-ec2f-4d3c-b6c8-1716587b7371"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 109])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: d623e625-b63d-4f90-882e-385f19de4874 (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } } } }, OmicronZoneConfig { id: ab512f01-b585-4697-95ff-95548582574a (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } } } }, OmicronZoneConfig { id: 1af22521-3df5-4f6c-b052-3f1bff202927 (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } } } }, OmicronZoneConfig { id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3 (service), filesystem_pool: Some(ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 6bb86fdc-adfa-4396-af16-3194717465e2 (service), filesystem_pool: Some(ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
9012024-12-13T22:28:03.249ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9022024-12-13T22:28:03.249ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9032024-12-13T22:28:03.284ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9042024-12-13T22:28:03.284ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }
9052024-12-13T22:28:03.317ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9062024-12-13T22:28:03.317ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }
9072024-12-13T22:28:03.352ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9082024-12-13T22:28:03.352ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9092024-12-13T22:28:03.385ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9102024-12-13T22:28:03.385ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }
9112024-12-13T22:28:03.417ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9122024-12-13T22:28:03.417ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }
9132024-12-13T22:28:03.450ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9142024-12-13T22:28:03.450ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
9152024-12-13T22:28:03.484ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9162024-12-13T22:28:03.484ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9172024-12-13T22:28:03.518ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
9182024-12-13T22:28:03.518ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:177
9192024-12-13T22:28:03.518ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:177
9202024-12-13T22:28:03.518ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
9212024-12-13T22:28:03.551ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
9222024-12-13T22:28:03.551ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
9232024-12-13T22:28:03.551ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
9242024-12-13T22:28:03.551ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
9252024-12-13T22:28:03.551ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
9262024-12-13T22:28:03.552ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3", TimeNotSynchronized), ("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a", TimeNotSynchronized), ("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2", TimeNotSynchronized), ("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874", TimeNotSynchronized), ("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 303873 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:49397 req_id = eda0f408-14b4-4e3e-a805-9eb0a8d4ed4e response_code = 500 uri = /omicron-zones
9272024-12-13T22:28:03.552ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "eda0f408-14b4-4e3e-a805-9eb0a8d4ed4e", "content-length": "124", "date": "Fri, 13 Dec 2024 22:28:03 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "eda0f408-14b4-4e3e-a805-9eb0a8d4ed4e" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 86.183928ms sled_address = [fd00:1122:3344:101::1]:12345
9282024-12-13T22:28:03.639ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 395a4d9e-1527-4351-8190-c1cd55768e7c (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 (service), filesystem_pool: Some(ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 89fc6686-a783-492f-b4f9-cc69952dca5b, kind: Service { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 }, name: Name("ntp-1f4a9057-ec2f-4d3c-b6c8-1716587b7371"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 109])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: d623e625-b63d-4f90-882e-385f19de4874 (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } } } }, OmicronZoneConfig { id: ab512f01-b585-4697-95ff-95548582574a (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } } } }, OmicronZoneConfig { id: 1af22521-3df5-4f6c-b052-3f1bff202927 (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } } } }, OmicronZoneConfig { id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3 (service), filesystem_pool: Some(ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 6bb86fdc-adfa-4396-af16-3194717465e2 (service), filesystem_pool: Some(ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
9292024-12-13T22:28:03.639ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9302024-12-13T22:28:03.639ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9312024-12-13T22:28:03.670ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9322024-12-13T22:28:03.670ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }
9332024-12-13T22:28:03.699ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9342024-12-13T22:28:03.699ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }
9352024-12-13T22:28:03.728ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9362024-12-13T22:28:03.728ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9372024-12-13T22:28:03.760ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9382024-12-13T22:28:03.760ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }
9392024-12-13T22:28:03.791ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9402024-12-13T22:28:03.791ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }
9412024-12-13T22:28:03.821ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9422024-12-13T22:28:03.821ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
9432024-12-13T22:28:03.853ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9442024-12-13T22:28:03.853ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9452024-12-13T22:28:03.884ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
9462024-12-13T22:28:03.884ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:177
9472024-12-13T22:28:03.885ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:177
9482024-12-13T22:28:03.885ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
9492024-12-13T22:28:03.915ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
9502024-12-13T22:28:03.915ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
9512024-12-13T22:28:03.915ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
9522024-12-13T22:28:03.915ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
9532024-12-13T22:28:03.915ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
9542024-12-13T22:28:03.915ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874", TimeNotSynchronized), ("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a", TimeNotSynchronized), ("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927", TimeNotSynchronized), ("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2", TimeNotSynchronized), ("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 276887 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:49397 req_id = c4e73552-24ef-4b00-9b46-733f8850faf6 response_code = 500 uri = /omicron-zones
9552024-12-13T22:28:03.915ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "c4e73552-24ef-4b00-9b46-733f8850faf6", "content-length": "124", "date": "Fri, 13 Dec 2024 22:28:03 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "c4e73552-24ef-4b00-9b46-733f8850faf6" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 255.055519ms sled_address = [fd00:1122:3344:101::1]:12345
9562024-12-13T22:28:04.171ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 395a4d9e-1527-4351-8190-c1cd55768e7c (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 (service), filesystem_pool: Some(ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 89fc6686-a783-492f-b4f9-cc69952dca5b, kind: Service { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 }, name: Name("ntp-1f4a9057-ec2f-4d3c-b6c8-1716587b7371"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 109])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: d623e625-b63d-4f90-882e-385f19de4874 (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } } } }, OmicronZoneConfig { id: ab512f01-b585-4697-95ff-95548582574a (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } } } }, OmicronZoneConfig { id: 1af22521-3df5-4f6c-b052-3f1bff202927 (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } } } }, OmicronZoneConfig { id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3 (service), filesystem_pool: Some(ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 6bb86fdc-adfa-4396-af16-3194717465e2 (service), filesystem_pool: Some(ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
9572024-12-13T22:28:04.171ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9582024-12-13T22:28:04.171ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9592024-12-13T22:28:04.205ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9602024-12-13T22:28:04.205ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }
9612024-12-13T22:28:04.238ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9622024-12-13T22:28:04.238ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }
9632024-12-13T22:28:04.270ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9642024-12-13T22:28:04.270ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9652024-12-13T22:28:04.302ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9662024-12-13T22:28:04.302ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }
9672024-12-13T22:28:04.336ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9682024-12-13T22:28:04.336ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }
9692024-12-13T22:28:04.368ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9702024-12-13T22:28:04.368ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
9712024-12-13T22:28:04.401ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9722024-12-13T22:28:04.401ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9732024-12-13T22:28:04.433ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
9742024-12-13T22:28:04.433ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:177
9752024-12-13T22:28:04.433ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:177
9762024-12-13T22:28:04.433ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
9772024-12-13T22:28:04.468ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
9782024-12-13T22:28:04.468ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
9792024-12-13T22:28:04.468ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
9802024-12-13T22:28:04.468ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
9812024-12-13T22:28:04.468ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
9822024-12-13T22:28:04.468ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a", TimeNotSynchronized), ("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927", TimeNotSynchronized), ("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2", TimeNotSynchronized), ("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874", TimeNotSynchronized), ("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 298047 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:49397 req_id = 781bdfbb-03d3-4be0-b20c-884908054aaa response_code = 500 uri = /omicron-zones
9832024-12-13T22:28:04.468ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "781bdfbb-03d3-4be0-b20c-884908054aaa", "content-length": "124", "date": "Fri, 13 Dec 2024 22:28:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "781bdfbb-03d3-4be0-b20c-884908054aaa" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 286.472643ms sled_address = [fd00:1122:3344:101::1]:12345
9842024-12-13T22:28:04.756ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 395a4d9e-1527-4351-8190-c1cd55768e7c (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 (service), filesystem_pool: Some(ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 89fc6686-a783-492f-b4f9-cc69952dca5b, kind: Service { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 }, name: Name("ntp-1f4a9057-ec2f-4d3c-b6c8-1716587b7371"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 109])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: d623e625-b63d-4f90-882e-385f19de4874 (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } } } }, OmicronZoneConfig { id: ab512f01-b585-4697-95ff-95548582574a (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } } } }, OmicronZoneConfig { id: 1af22521-3df5-4f6c-b052-3f1bff202927 (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } } } }, OmicronZoneConfig { id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3 (service), filesystem_pool: Some(ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 6bb86fdc-adfa-4396-af16-3194717465e2 (service), filesystem_pool: Some(ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
9852024-12-13T22:28:04.756ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9862024-12-13T22:28:04.756ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }
9872024-12-13T22:28:04.792ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9882024-12-13T22:28:04.792ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }
9892024-12-13T22:28:04.824ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
9902024-12-13T22:28:04.824ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }
9912024-12-13T22:28:04.854ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9922024-12-13T22:28:04.855ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }
9932024-12-13T22:28:04.887ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9942024-12-13T22:28:04.887ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }
9952024-12-13T22:28:04.919ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9962024-12-13T22:28:04.919ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }
9972024-12-13T22:28:04.952ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
9982024-12-13T22:28:04.952ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
9992024-12-13T22:28:04.984ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10002024-12-13T22:28:04.984ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10012024-12-13T22:28:05.017ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10022024-12-13T22:28:05.017ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:177
10032024-12-13T22:28:05.017ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:177
10042024-12-13T22:28:05.017ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10052024-12-13T22:28:05.049ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
10062024-12-13T22:28:05.049ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
10072024-12-13T22:28:05.049ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
10082024-12-13T22:28:05.049ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
10092024-12-13T22:28:05.049ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
10102024-12-13T22:28:05.049ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927", TimeNotSynchronized), ("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874", TimeNotSynchronized), ("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2", TimeNotSynchronized), ("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a", TimeNotSynchronized), ("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 294000 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:49397 req_id = 3218a8c7-bec5-49b5-a2da-a9aef75f5d07 response_code = 500 uri = /omicron-zones
10112024-12-13T22:28:05.050ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "3218a8c7-bec5-49b5-a2da-a9aef75f5d07", "content-length": "124", "date": "Fri, 13 Dec 2024 22:28:04 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "3218a8c7-bec5-49b5-a2da-a9aef75f5d07" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 351.274539ms sled_address = [fd00:1122:3344:101::1]:12345
10122024-12-13T22:28:05.401ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 395a4d9e-1527-4351-8190-c1cd55768e7c (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 (service), filesystem_pool: Some(ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 89fc6686-a783-492f-b4f9-cc69952dca5b, kind: Service { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 }, name: Name("ntp-1f4a9057-ec2f-4d3c-b6c8-1716587b7371"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 109])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: d623e625-b63d-4f90-882e-385f19de4874 (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } } } }, OmicronZoneConfig { id: ab512f01-b585-4697-95ff-95548582574a (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } } } }, OmicronZoneConfig { id: 1af22521-3df5-4f6c-b052-3f1bff202927 (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } } } }, OmicronZoneConfig { id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3 (service), filesystem_pool: Some(ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 6bb86fdc-adfa-4396-af16-3194717465e2 (service), filesystem_pool: Some(ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
10132024-12-13T22:28:05.401ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10142024-12-13T22:28:05.402ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10152024-12-13T22:28:05.436ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10162024-12-13T22:28:05.436ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }
10172024-12-13T22:28:05.468ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10182024-12-13T22:28:05.468ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }
10192024-12-13T22:28:05.501ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10202024-12-13T22:28:05.501ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10212024-12-13T22:28:05.506ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 358.467756ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
10222024-12-13T22:28:05.534ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10232024-12-13T22:28:05.534ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }
10242024-12-13T22:28:05.567ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10252024-12-13T22:28:05.567ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }
10262024-12-13T22:28:05.599ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10272024-12-13T22:28:05.599ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
10282024-12-13T22:28:05.633ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10292024-12-13T22:28:05.633ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10302024-12-13T22:28:05.665ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10312024-12-13T22:28:05.665ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:177
10322024-12-13T22:28:05.665ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:177
10332024-12-13T22:28:05.665ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10342024-12-13T22:28:05.697ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
10352024-12-13T22:28:05.698ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
10362024-12-13T22:28:05.698ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
10372024-12-13T22:28:05.698ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
10382024-12-13T22:28:05.698ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
10392024-12-13T22:28:05.698ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a", TimeNotSynchronized), ("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927", TimeNotSynchronized), ("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2", TimeNotSynchronized), ("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874", TimeNotSynchronized), ("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 297110 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:49397 req_id = 5779b3b8-1f5b-4024-847d-86f32eff6a35 response_code = 500 uri = /omicron-zones
10402024-12-13T22:28:05.698ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "5779b3b8-1f5b-4024-847d-86f32eff6a35", "content-length": "124", "date": "Fri, 13 Dec 2024 22:28:05 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "5779b3b8-1f5b-4024-847d-86f32eff6a35" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 184.663277ms sled_address = [fd00:1122:3344:101::1]:12345
10412024-12-13T22:28:05.896ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 395a4d9e-1527-4351-8190-c1cd55768e7c (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 (service), filesystem_pool: Some(ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 89fc6686-a783-492f-b4f9-cc69952dca5b, kind: Service { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 }, name: Name("ntp-1f4a9057-ec2f-4d3c-b6c8-1716587b7371"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 109])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: d623e625-b63d-4f90-882e-385f19de4874 (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } } } }, OmicronZoneConfig { id: ab512f01-b585-4697-95ff-95548582574a (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } } } }, OmicronZoneConfig { id: 1af22521-3df5-4f6c-b052-3f1bff202927 (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } } } }, OmicronZoneConfig { id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3 (service), filesystem_pool: Some(ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 6bb86fdc-adfa-4396-af16-3194717465e2 (service), filesystem_pool: Some(ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
10422024-12-13T22:28:05.897ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10432024-12-13T22:28:05.897ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10442024-12-13T22:28:05.932ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10452024-12-13T22:28:05.932ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }
10462024-12-13T22:28:05.964ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10472024-12-13T22:28:05.964ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }
10482024-12-13T22:28:05.993ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10492024-12-13T22:28:05.993ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10502024-12-13T22:28:06.025ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10512024-12-13T22:28:06.025ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }
10522024-12-13T22:28:06.058ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10532024-12-13T22:28:06.058ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }
10542024-12-13T22:28:06.092ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10552024-12-13T22:28:06.092ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
10562024-12-13T22:28:06.124ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10572024-12-13T22:28:06.124ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10582024-12-13T22:28:06.157ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10592024-12-13T22:28:06.157ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:177
10602024-12-13T22:28:06.157ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:177
10612024-12-13T22:28:06.157ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10622024-12-13T22:28:06.189ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
10632024-12-13T22:28:06.189ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
10642024-12-13T22:28:06.189ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
10652024-12-13T22:28:06.189ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
10662024-12-13T22:28:06.189ZWARNSledAgent (ServiceManager): Zone failed to start file = sled-agent/src/services.rs:3314 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
10672024-12-13T22:28:06.189ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to initialize zones: [("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2", TimeNotSynchronized), ("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874", TimeNotSynchronized), ("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a", TimeNotSynchronized), ("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927", TimeNotSynchronized), ("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3", TimeNotSynchronized)] file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 293155 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:49397 req_id = 052761e4-1934-4617-9473-866896f29ee3 response_code = 500 uri = /omicron-zones
10682024-12-13T22:28:06.189ZWARNSledAgent (RSS): failed to initialize Omicron zones error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "052761e4-1934-4617-9473-866896f29ee3", "content-length": "124", "date": "Fri, 13 Dec 2024 22:28:05 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "052761e4-1934-4617-9473-866896f29ee3" } file = sled-agent/src/rack_setup/service.rs:578 retry_after = 452.406295ms sled_address = [fd00:1122:3344:101::1]:12345
10692024-12-13T22:28:06.641ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(4), zones: [OmicronZoneConfig { id: 395a4d9e-1527-4351-8190-c1cd55768e7c (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 (service), filesystem_pool: Some(ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 89fc6686-a783-492f-b4f9-cc69952dca5b, kind: Service { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 }, name: Name("ntp-1f4a9057-ec2f-4d3c-b6c8-1716587b7371"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 109])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: d623e625-b63d-4f90-882e-385f19de4874 (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } } } }, OmicronZoneConfig { id: ab512f01-b585-4697-95ff-95548582574a (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } } } }, OmicronZoneConfig { id: 1af22521-3df5-4f6c-b052-3f1bff202927 (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } } } }, OmicronZoneConfig { id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3 (service), filesystem_pool: Some(ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 6bb86fdc-adfa-4396-af16-3194717465e2 (service), filesystem_pool: Some(ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
10702024-12-13T22:28:06.641ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10712024-12-13T22:28:06.641ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }
10722024-12-13T22:28:06.676ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10732024-12-13T22:28:06.676ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }
10742024-12-13T22:28:06.708ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
10752024-12-13T22:28:06.708ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }
10762024-12-13T22:28:06.740ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10772024-12-13T22:28:06.740ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10782024-12-13T22:28:06.773ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10792024-12-13T22:28:06.773ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }
10802024-12-13T22:28:06.806ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10812024-12-13T22:28:06.806ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }
10822024-12-13T22:28:06.840ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10832024-12-13T22:28:06.840ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
10842024-12-13T22:28:06.875ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
10852024-12-13T22:28:06.875ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }
10862024-12-13T22:28:06.908ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10872024-12-13T22:28:06.908ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:177
10882024-12-13T22:28:06.908ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:177
10892024-12-13T22:28:06.908ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10902024-12-13T22:28:06.990ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10912024-12-13T22:28:06.990ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10922024-12-13T22:28:06.990ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10932024-12-13T22:28:06.990ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10942024-12-13T22:28:06.990ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
10952024-12-13T22:28:07.079ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a file = illumos-utils/src/zone.rs:325
10962024-12-13T22:28:07.083ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2 file = illumos-utils/src/zone.rs:325
10972024-12-13T22:28:07.087ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927 file = illumos-utils/src/zone.rs:325
10982024-12-13T22:28:07.091ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3 file = illumos-utils/src/zone.rs:325
10992024-12-13T22:28:07.095ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874 file = illumos-utils/src/zone.rs:325
11002024-12-13T22:28:07.100ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a file = illumos-utils/src/zone.rs:363
11012024-12-13T22:28:07.106ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2 file = illumos-utils/src/zone.rs:363
11022024-12-13T22:28:07.111ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927 file = illumos-utils/src/zone.rs:363
11032024-12-13T22:28:07.115ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3 file = illumos-utils/src/zone.rs:363
11042024-12-13T22:28:07.119ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874 file = illumos-utils/src/zone.rs:363
11052024-12-13T22:28:09.900ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService5'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::4'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='ab512f01-b585-4697-95ff-95548582574a'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::4]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::4]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11062024-12-13T22:28:09.900ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
11072024-12-13T22:28:09.916ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService6'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='6bb86fdc-adfa-4396-af16-3194717465e2'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::7]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::7]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11082024-12-13T22:28:09.916ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService8'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='abb00af3-5d1a-42d8-8103-7ee7bc3835d3'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::6]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::6]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11092024-12-13T22:28:09.916ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11102024-12-13T22:28:09.922ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11112024-12-13T22:28:10.043ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService9'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::3'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='d623e625-b63d-4f90-882e-385f19de4874'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::3]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::3]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11122024-12-13T22:28:10.043ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
11132024-12-13T22:28:10.056ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a") failed: Property not found. retry in 67.350913ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
11142024-12-13T22:28:10.064ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 53.199459ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11152024-12-13T22:28:10.071ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 65.072899ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11162024-12-13T22:28:10.117ZINFOSledAgent (ServiceManager): Profile for oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService7'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/cockroachdb"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="oxide/cockroach-admin"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="zone_id" value='1af22521-3df5-4f6c-b052-3f1bff202927'/> <propval type="astring" name="cockroach_address" value='[fd00:1122:3344:101::5]:32221'/> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::5]:32222'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
11172024-12-13T22:28:10.117ZINFOSledAgent (ServiceManager): Booting oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
11182024-12-13T22:28:10.224ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 100.606307ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11192024-12-13T22:28:10.435ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a") failed: Property not found. retry in 121.30903ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
11202024-12-13T22:28:10.479ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 127.225026ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11212024-12-13T22:28:10.507ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 171.551259ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11222024-12-13T22:28:10.543ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 28.916635ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
11232024-12-13T22:28:10.616ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a") failed: Property not found. retry in 242.983094ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
11242024-12-13T22:28:10.624ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 113.69796ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
11252024-12-13T22:28:10.704ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 281.042094ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11262024-12-13T22:28:10.737ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927") failed: Property not found. retry in 58.739673ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
11272024-12-13T22:28:10.980ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 366.936547ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11282024-12-13T22:28:11.622ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 212.37583ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
11292024-12-13T22:28:11.828ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a") failed: Property not found. retry in 216.692987ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
11302024-12-13T22:28:11.870ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 226.960686ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11312024-12-13T22:28:11.892ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927") failed: Property not found. retry in 135.263584ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
11322024-12-13T22:28:11.924ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 549.50958ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11332024-12-13T22:28:11.947ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 275.358576ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
11342024-12-13T22:28:12.044ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927") failed: Property not found. retry in 141.691931ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
11352024-12-13T22:28:12.093ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a") failed: Property not found. retry in 917.19929ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
11362024-12-13T22:28:12.135ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 603.556378ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11372024-12-13T22:28:12.203ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927") failed: Property not found. retry in 280.19888ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
11382024-12-13T22:28:12.237ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 467.958497ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
11392024-12-13T22:28:12.488ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 1.297951122s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11402024-12-13T22:28:12.501ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927") failed: Property not found. retry in 622.554236ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
11412024-12-13T22:28:12.717ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 544.804514ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
11422024-12-13T22:28:12.750ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 851.755418ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11432024-12-13T22:28:13.019ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a") failed: Property not found. retry in 895.216584ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
11442024-12-13T22:28:13.133ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927") failed: Property not found. retry in 1.22918052s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
11452024-12-13T22:28:13.272ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 547.324297ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
11462024-12-13T22:28:13.613ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 979.754168ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11472024-12-13T22:28:13.795ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 751.322902ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11482024-12-13T22:28:13.828ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 742.343246ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
11492024-12-13T22:28:13.924ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a") failed: Property not found. retry in 696.892391ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
11502024-12-13T22:28:14.372ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927") failed: Property not found. retry in 595.81487ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
11512024-12-13T22:28:14.556ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 838.953784ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11522024-12-13T22:28:14.580ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 1.37162685s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
11532024-12-13T22:28:14.601ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 1.034732272s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11542024-12-13T22:28:14.631ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a") failed: Property not found. retry in 1.286226974s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
11552024-12-13T22:28:14.977ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927") failed: Property not found. retry in 1.068690296s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
11562024-12-13T22:28:15.405ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 1.022133285s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11572024-12-13T22:28:15.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 1.123383979s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11582024-12-13T22:28:15.928ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a") failed: Property not found. retry in 725.607335ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
11592024-12-13T22:28:15.961ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 1.414761548s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
11602024-12-13T22:28:16.056ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927") failed: Property not found. retry in 1.37849727s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
11612024-12-13T22:28:16.439ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 970.584674ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11622024-12-13T22:28:16.663ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a") failed: Property not found. retry in 1.444774767s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
11632024-12-13T22:28:16.779ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 1.180375957s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11642024-12-13T22:28:17.385ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 716.891168ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
11652024-12-13T22:28:17.419ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 589.981515ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11662024-12-13T22:28:17.445ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927") failed: Property not found. retry in 1.496152023s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
11672024-12-13T22:28:17.969ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 887.466388ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11682024-12-13T22:28:18.018ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 589.395852ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11692024-12-13T22:28:18.113ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 972.987941ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
11702024-12-13T22:28:18.121ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a") failed: Property not found. retry in 823.487427ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
11712024-12-13T22:28:18.618ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 1.061434722s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11722024-12-13T22:28:18.868ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 576.708917ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11732024-12-13T22:28:18.952ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927") failed: Property not found. retry in 1.342956495s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
11742024-12-13T22:28:18.961ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a") failed: Property not found. retry in 1.40128159s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
11752024-12-13T22:28:19.096ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 651.480647ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
11762024-12-13T22:28:19.456ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 1.47475138s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11772024-12-13T22:28:19.689ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 501.56798ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11782024-12-13T22:28:19.757ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 1.256886112s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
11792024-12-13T22:28:20.200ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 646.539718ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11802024-12-13T22:28:20.305ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927") failed: Property not found. retry in 708.332784ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
11812024-12-13T22:28:20.371ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a") failed: Property not found. retry in 882.06403ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
11822024-12-13T22:28:20.858ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 1.045115966s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11832024-12-13T22:28:20.885ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 294.692586ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
11842024-12-13T22:28:20.943ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 588.484736ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11852024-12-13T22:28:21.024ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927") failed: Property not found. retry in 1.217730219s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
11862024-12-13T22:28:21.034ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 1.323991208s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
11872024-12-13T22:28:21.264ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a") failed: Property not found. retry in 547.892537ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
11882024-12-13T22:28:21.541ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 562.168205ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11892024-12-13T22:28:21.821ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a") failed: Property not found. retry in 1.007465275s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
11902024-12-13T22:28:21.913ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 1.150117847s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11912024-12-13T22:28:22.113ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 1.260943145s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11922024-12-13T22:28:22.252ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927") failed: Property not found. retry in 1.287872157s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
11932024-12-13T22:28:22.368ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 1.259621795s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
11942024-12-13T22:28:22.838ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a") failed: Property not found. retry in 1.156767504s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
11952024-12-13T22:28:23.073ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 525.397825ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11962024-12-13T22:28:23.383ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 1.077915815s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
11972024-12-13T22:28:23.550ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927") failed: Property not found. retry in 1.351268962s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
11982024-12-13T22:28:23.607ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3") failed: Property not found. retry in 1.489969389s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
11992024-12-13T22:28:23.638ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 555.09945ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
12002024-12-13T22:28:24.006ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a") failed: Property not found. retry in 1.384982075s file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
12012024-12-13T22:28:24.203ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874") failed: Property not found. retry in 706.236358ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
12022024-12-13T22:28:24.472ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2") failed: Property not found. retry in 827.533712ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
12032024-12-13T22:28:24.911ZWARNSledAgent (ServiceManager): wait for service Some("oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927") failed: Property not found. retry in 986.611396ms file = illumos-utils/src/svc.rs:36 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
12042024-12-13T22:28:25.922ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_cockroachdb_ab512f01-b585-4697-95ff-95548582574a
12052024-12-13T22:28:25.922ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_cockroachdb_6bb86fdc-adfa-4396-af16-3194717465e2
12062024-12-13T22:28:25.922ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927
12072024-12-13T22:28:25.922ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_cockroachdb_abb00af3-5d1a-42d8-8103-7ee7bc3835d3
12082024-12-13T22:28:25.922ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_cockroachdb_d623e625-b63d-4f90-882e-385f19de4874
12092024-12-13T22:28:25.922ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
12102024-12-13T22:28:25.922ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
12112024-12-13T22:28:25.923ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 19294233 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:49397 req_id = 5a5ff9a6-d4a8-48a6-a564-a7afc0d27e3b response_code = 204 uri = /omicron-zones
12122024-12-13T22:28:25.958ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:64407
12132024-12-13T22:28:25.988ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService7/omicron6 in oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService7/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 30453 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64407 req_id = 5d58b006-eff9-44d3-85f1-cd7e86fd3dfe response_code = 500 uri = /cockroachdb
12142024-12-13T22:28:25.988ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "5d58b006-eff9-44d3-85f1-cd7e86fd3dfe", "content-length": "124", "date": "Fri, 13 Dec 2024 22:28:25 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "5d58b006-eff9-44d3-85f1-cd7e86fd3dfe" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 101.134429ms
12152024-12-13T22:28:26.119ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService7/omicron6 in oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService7/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 28181 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64407 req_id = eb1d1bfe-f653-4500-b45d-cbdfac9b8c4d response_code = 500 uri = /cockroachdb
12162024-12-13T22:28:26.119ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "eb1d1bfe-f653-4500-b45d-cbdfac9b8c4d", "content-length": "124", "date": "Fri, 13 Dec 2024 22:28:25 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "eb1d1bfe-f653-4500-b45d-cbdfac9b8c4d" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 179.969723ms
12172024-12-13T22:28:26.326ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService7/omicron6 in oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService7/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 25500 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64407 req_id = 0d786454-312d-4324-ab1f-f110ae2a3059 response_code = 500 uri = /cockroachdb
12182024-12-13T22:28:26.326ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "0d786454-312d-4324-ab1f-f110ae2a3059", "content-length": "124", "date": "Fri, 13 Dec 2024 22:28:25 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "0d786454-312d-4324-ab1f-f110ae2a3059" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 102.889996ms
12192024-12-13T22:28:26.458ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService7/omicron6 in oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService7/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 27928 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64407 req_id = cf21e7cf-82d5-4b02-ac48-1556d49b6daf response_code = 500 uri = /cockroachdb
12202024-12-13T22:28:26.458ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "cf21e7cf-82d5-4b02-ac48-1556d49b6daf", "content-length": "124", "date": "Fri, 13 Dec 2024 22:28:25 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "cf21e7cf-82d5-4b02-ac48-1556d49b6daf" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 133.392993ms
12212024-12-13T22:28:26.619ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService7/omicron6 in oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService7/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 26497 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64407 req_id = 027f47ca-1e3b-4108-86ef-fe8c9d962e1f response_code = 500 uri = /cockroachdb
12222024-12-13T22:28:26.620ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "027f47ca-1e3b-4108-86ef-fe8c9d962e1f", "content-length": "124", "date": "Fri, 13 Dec 2024 22:28:25 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "027f47ca-1e3b-4108-86ef-fe8c9d962e1f" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 277.388128ms
12232024-12-13T22:28:26.925ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService7/omicron6 in oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService7/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 27478 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64407 req_id = fd1df90f-d8bc-48d0-9b84-4350af4fd94b response_code = 500 uri = /cockroachdb
12242024-12-13T22:28:26.925ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "fd1df90f-d8bc-48d0-9b84-4350af4fd94b", "content-length": "124", "date": "Fri, 13 Dec 2024 22:28:26 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "fd1df90f-d8bc-48d0-9b84-4350af4fd94b" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 210.311562ms
12252024-12-13T22:28:27.167ZINFOSledAgent (dropshot (SledAgent)): request completed error_message_external = Internal Server Error error_message_internal = Failed to get address: Failed to get address for name oxControlService7/omicron6 in oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927: Zone execution error: Command [/usr/sbin/zlogin oxz_cockroachdb_1af22521-3df5-4f6c-b052-3f1bff202927 /usr/sbin/ipadm show-addr -p -o ADDR oxControlService7/omicron6] executed and failed with status: exit status: 1 stdout: \n stderr: ipadm: Could not get address: Object not found file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:851 latency_us = 29824 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64407 req_id = 6a0b1c69-bbab-4203-85ea-daf78466ddfc response_code = 500 uri = /cockroachdb
12262024-12-13T22:28:27.167ZWARNSledAgent (RSS): Failed to initialize CockroachDB error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "6a0b1c69-bbab-4203-85ea-daf78466ddfc", "content-length": "124", "date": "Fri, 13 Dec 2024 22:28:26 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "6a0b1c69-bbab-4203-85ea-daf78466ddfc" } file = sled-agent/src/rack_setup/service.rs:1157 retry_after = 238.081022ms
12272024-12-13T22:28:27.438ZINFOSledAgent (ServiceManager): Initializing CRDB Cluster - sending request to [fd00:1122:3344:101::5]:32221 file = sled-agent/src/services.rs:3743
12282024-12-13T22:28:52.147ZINFOSledAgent (ServiceManager): Formatting CRDB file = sled-agent/src/services.rs:3761
12292024-12-13T22:28:56.718ZINFOSledAgent (ServiceManager): Formatting CRDB - Completed file = sled-agent/src/services.rs:3784
12302024-12-13T22:28:56.718ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 29312591 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::1]:64407 req_id = c61c6c77-0deb-451d-9ae5-5b046707ec7b response_code = 204 uri = /cockroachdb
12312024-12-13T22:28:56.721ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:42630
12322024-12-13T22:28:56.732ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 502.248743ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
12332024-12-13T22:28:56.766ZINFOSledAgent (RSS): attempting to set up sled's Omicron zones: OmicronZonesConfig { generation: Generation(5), zones: [OmicronZoneConfig { id: 395a4d9e-1527-4351-8190-c1cd55768e7c (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } }, http_address: [fd00:1122:3344:1::1]:5353, dns_address: [fd00:1122:3344:1::1]:53, gz_address: fd00:1122:3344:1::2, gz_address_index: 0 } }, OmicronZoneConfig { id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } }, http_address: [fd00:1122:3344:2::1]:5353, dns_address: [fd00:1122:3344:2::1]:53, gz_address: fd00:1122:3344:2::2, gz_address_index: 1 } }, OmicronZoneConfig { id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: InternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } }, http_address: [fd00:1122:3344:3::1]:5353, dns_address: [fd00:1122:3344:3::1]:53, gz_address: fd00:1122:3344:3::2, gz_address_index: 2 } }, OmicronZoneConfig { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 (service), filesystem_pool: Some(ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }), zone_type: BoundaryNtp { address: [fd00:1122:3344:101::1b]:123, ntp_servers: ["0.pool.ntp.org"], dns_servers: [1.1.1.1, 9.9.9.9], domain: None, nic: NetworkInterface { id: 89fc6686-a783-492f-b4f9-cc69952dca5b, kind: Service { id: 1f4a9057-ec2f-4d3c-b6c8-1716587b7371 }, name: Name("ntp-1f4a9057-ec2f-4d3c-b6c8-1716587b7371"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 216, 109])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } } }, OmicronZoneConfig { id: d623e625-b63d-4f90-882e-385f19de4874 (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::3]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } } } }, OmicronZoneConfig { id: ab512f01-b585-4697-95ff-95548582574a (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::4]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } } } }, OmicronZoneConfig { id: 1af22521-3df5-4f6c-b052-3f1bff202927 (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::5]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } } } }, OmicronZoneConfig { id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3 (service), filesystem_pool: Some(ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::6]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 6bb86fdc-adfa-4396-af16-3194717465e2 (service), filesystem_pool: Some(ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }), zone_type: CockroachDb { address: [fd00:1122:3344:101::7]:32221, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External } } } }, OmicronZoneConfig { id: 3e0d656e-3552-43ff-8e95-5ed82471e0ae (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } }, http_address: [fd00:1122:3344:101::8]:5353, dns_address: 10.151.2.164:53, nic: NetworkInterface { id: 0e0028b4-c96e-4259-b0d9-2dd1e967b57e, kind: Service { id: 3e0d656e-3552-43ff-8e95-5ed82471e0ae }, name: Name("external-dns-3e0d656e-3552-43ff-8e95-5ed82471e0ae"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 156])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: 0ef774ca-06b2-40a8-a632-7304d594c07a (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: ExternalDns { dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } }, http_address: [fd00:1122:3344:101::9]:5353, dns_address: 10.151.2.165:53, nic: NetworkInterface { id: 9fbc6464-5658-4709-85f5-cba4ad21c9de, kind: Service { id: 0ef774ca-06b2-40a8-a632-7304d594c07a }, name: Name("external-dns-0ef774ca-06b2-40a8-a632-7304d594c07a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 39])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } } }, OmicronZoneConfig { id: b2389da8-cde2-4174-a8e1-a4dabd361b80 (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::a]:12221, external_ip: 10.151.2.166, nic: NetworkInterface { id: be9ea2e0-22c4-425c-9afb-1597ead9d56b, kind: Service { id: b2389da8-cde2-4174-a8e1-a4dabd361b80 }, name: Name("nexus-b2389da8-cde2-4174-a8e1-a4dabd361b80"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 223, 94])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: ef47bd05-ad7e-42a3-8257-dec06acd2b93 (service), filesystem_pool: Some(ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::b]:12221, external_ip: 10.151.2.167, nic: NetworkInterface { id: bb1ce312-163f-477c-9052-27f110965501, kind: Service { id: ef47bd05-ad7e-42a3-8257-dec06acd2b93 }, name: Name("nexus-ef47bd05-ad7e-42a3-8257-dec06acd2b93"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 50])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 7e3d78a7-b6c3-4636-b217-b0a79485c981 (service), filesystem_pool: Some(ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }), zone_type: Nexus { internal_address: [fd00:1122:3344:101::c]:12221, external_ip: 10.151.2.168, nic: NetworkInterface { id: e47aafa9-7219-497b-8e24-b813038ee602, kind: Service { id: 7e3d78a7-b6c3-4636-b217-b0a79485c981 }, name: Name("nexus-7e3d78a7-b6c3-4636-b217-b0a79485c981"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 163])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, external_tls: true, external_dns_servers: [1.1.1.1, 9.9.9.9] } }, OmicronZoneConfig { id: 689eb899-96a8-4e42-a5ca-864d1c68fdc9 (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: Oximeter { address: [fd00:1122:3344:101::d]:12223 } }, OmicronZoneConfig { id: 2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2 (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: Clickhouse { address: [fd00:1122:3344:101::e]:8123, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } } } }, OmicronZoneConfig { id: fef903b4-5e26-41cf-9ef8-fdf744c1ce8c (service), filesystem_pool: Some(ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::f]:17000 } }, OmicronZoneConfig { id: 0dd7b0ec-9c6d-4813-9a5e-cd184cdec869 (service), filesystem_pool: Some(ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::10]:17000 } }, OmicronZoneConfig { id: 128c56c1-c87f-4290-bec9-772d8c30d934 (service), filesystem_pool: Some(ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }), zone_type: CruciblePantry { address: [fd00:1122:3344:101::11]:17000 } }, OmicronZoneConfig { id: ceccfc26-d275-4f22-bea6-d0c86043e1f0 (service), filesystem_pool: Some(ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::12]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External } } } }, OmicronZoneConfig { id: acec6218-e24c-44eb-99aa-6d30724f14ab (service), filesystem_pool: Some(ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::13]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External } } } }, OmicronZoneConfig { id: d64a1818-8633-453a-b13f-88ca4e89a7c4 (service), filesystem_pool: Some(ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::14]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External } } } }, OmicronZoneConfig { id: 75f07622-aad2-4d27-9e07-2856e4fb4722 (service), filesystem_pool: Some(ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::15]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External } } } }, OmicronZoneConfig { id: 35d09a11-5d25-47f0-a6f4-341cbb551dd5 (service), filesystem_pool: Some(ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::16]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External } } } }, OmicronZoneConfig { id: 79db7e3d-21d6-43fe-bcec-aa471a9e012c (service), filesystem_pool: Some(ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::17]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External } } } }, OmicronZoneConfig { id: 694b4960-fc0f-482e-8da6-0ffd414578b4 (service), filesystem_pool: Some(ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::18]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External } } } }, OmicronZoneConfig { id: 45856571-2c50-487e-9962-2d48d24e4919 (service), filesystem_pool: Some(ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::19]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External } } } }, OmicronZoneConfig { id: 74b96c49-e4d9-4174-9313-533584a77cb5 (service), filesystem_pool: Some(ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }), zone_type: Crucible { address: [fd00:1122:3344:101::1a]:32345, dataset: OmicronZoneDataset { pool_name: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External } } } }] } file = sled-agent/src/rack_setup/service.rs:541 sled_address = [fd00:1122:3344:101::1]:12345
12342024-12-13T22:28:56.766ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::1]:48088
12352024-12-13T22:28:56.768ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
12362024-12-13T22:28:56.768ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 395a4d9e-1527-4351-8190-c1cd55768e7c, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: InternalDns }, responder: ... }
12372024-12-13T22:28:56.820ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
12382024-12-13T22:28:56.821ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2bddd59f-8fa2-4b3b-ae1a-b309cf00641b, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: InternalDns }, responder: ... }
12392024-12-13T22:28:56.871ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
12402024-12-13T22:28:56.871ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: b14af4bc-de06-49b6-a6f5-2ef07a34b0dc, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: InternalDns }, responder: ... }
12412024-12-13T22:28:56.918ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
12422024-12-13T22:28:56.918ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: d623e625-b63d-4f90-882e-385f19de4874, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Cockroach }, responder: ... }
12432024-12-13T22:28:56.970ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
12442024-12-13T22:28:56.970ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ab512f01-b585-4697-95ff-95548582574a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Cockroach }, responder: ... }
12452024-12-13T22:28:57.021ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
12462024-12-13T22:28:57.021ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 1af22521-3df5-4f6c-b052-3f1bff202927, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Cockroach }, responder: ... }
12472024-12-13T22:28:57.064ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
12482024-12-13T22:28:57.064ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: abb00af3-5d1a-42d8-8103-7ee7bc3835d3, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Cockroach }, responder: ... }
12492024-12-13T22:28:57.114ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }) file = sled-storage/src/manager.rs:603
12502024-12-13T22:28:57.114ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 6bb86fdc-adfa-4396-af16-3194717465e2, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Cockroach }, responder: ... }
12512024-12-13T22:28:57.162ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 3e0d656e-3552-43ff-8e95-5ed82471e0ae, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
12522024-12-13T22:28:57.162ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 3e0d656e-3552-43ff-8e95-5ed82471e0ae, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: ExternalDns }, responder: ... }
12532024-12-13T22:28:57.208ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 0ef774ca-06b2-40a8-a632-7304d594c07a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: ExternalDns }, responder: ... }) file = sled-storage/src/manager.rs:603
12542024-12-13T22:28:57.208ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 0ef774ca-06b2-40a8-a632-7304d594c07a, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: ExternalDns }, responder: ... }
12552024-12-13T22:28:57.256ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Clickhouse }, responder: ... }) file = sled-storage/src/manager.rs:603
12562024-12-13T22:28:57.256ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Clickhouse }, responder: ... }
12572024-12-13T22:28:57.304ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: ceccfc26-d275-4f22-bea6-d0c86043e1f0, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12582024-12-13T22:28:57.304ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: ceccfc26-d275-4f22-bea6-d0c86043e1f0, dataset_name: DatasetName { pool_name: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, kind: Crucible }, responder: ... }
12592024-12-13T22:28:57.350ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: acec6218-e24c-44eb-99aa-6d30724f14ab, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12602024-12-13T22:28:57.350ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: acec6218-e24c-44eb-99aa-6d30724f14ab, dataset_name: DatasetName { pool_name: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, kind: Crucible }, responder: ... }
12612024-12-13T22:28:57.389ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: d64a1818-8633-453a-b13f-88ca4e89a7c4, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12622024-12-13T22:28:57.389ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: d64a1818-8633-453a-b13f-88ca4e89a7c4, dataset_name: DatasetName { pool_name: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, kind: Crucible }, responder: ... }
12632024-12-13T22:28:57.430ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 75f07622-aad2-4d27-9e07-2856e4fb4722, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12642024-12-13T22:28:57.430ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 75f07622-aad2-4d27-9e07-2856e4fb4722, dataset_name: DatasetName { pool_name: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, kind: Crucible }, responder: ... }
12652024-12-13T22:28:57.476ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 35d09a11-5d25-47f0-a6f4-341cbb551dd5, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12662024-12-13T22:28:57.476ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 35d09a11-5d25-47f0-a6f4-341cbb551dd5, dataset_name: DatasetName { pool_name: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, kind: Crucible }, responder: ... }
12672024-12-13T22:28:57.520ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 79db7e3d-21d6-43fe-bcec-aa471a9e012c, dataset_name: DatasetName { pool_name: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12682024-12-13T22:28:57.520ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 79db7e3d-21d6-43fe-bcec-aa471a9e012c, dataset_name: DatasetName { pool_name: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }, kind: Crucible }, responder: ... }
12692024-12-13T22:28:57.564ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 694b4960-fc0f-482e-8da6-0ffd414578b4, dataset_name: DatasetName { pool_name: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12702024-12-13T22:28:57.564ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 694b4960-fc0f-482e-8da6-0ffd414578b4, dataset_name: DatasetName { pool_name: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }, kind: Crucible }, responder: ... }
12712024-12-13T22:28:57.612ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 45856571-2c50-487e-9962-2d48d24e4919, dataset_name: DatasetName { pool_name: ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12722024-12-13T22:28:57.612ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 45856571-2c50-487e-9962-2d48d24e4919, dataset_name: DatasetName { pool_name: ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External }, kind: Crucible }, responder: ... }
12732024-12-13T22:28:57.656ZINFOSledAgent (StorageManager): Received NewFilesystem(NewFilesystemRequest { dataset_id: 74b96c49-e4d9-4174-9313-533584a77cb5, dataset_name: DatasetName { pool_name: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, kind: Crucible }, responder: ... }) file = sled-storage/src/manager.rs:603
12742024-12-13T22:28:57.657ZINFOSledAgent (StorageManager): add_dataset file = sled-storage/src/manager.rs:1379 request = NewFilesystemRequest { dataset_id: 74b96c49-e4d9-4174-9313-533584a77cb5, dataset_name: DatasetName { pool_name: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, kind: Crucible }, responder: ... }
12752024-12-13T22:28:57.702ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12762024-12-13T22:28:57.702ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:177
12772024-12-13T22:28:57.702ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:177
12782024-12-13T22:28:57.702ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12792024-12-13T22:28:57.833ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12802024-12-13T22:28:57.834ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12812024-12-13T22:28:57.834ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12822024-12-13T22:28:57.886ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12832024-12-13T22:28:57.886ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12842024-12-13T22:28:57.886ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12852024-12-13T22:28:57.886ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12862024-12-13T22:28:57.886ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12872024-12-13T22:28:57.888ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12882024-12-13T22:28:57.901ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12892024-12-13T22:28:58.099ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12902024-12-13T22:28:58.099ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12912024-12-13T22:28:58.099ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12922024-12-13T22:28:58.099ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12932024-12-13T22:28:58.100ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12942024-12-13T22:28:58.116ZINFOSledAgent (ServiceManager): Resolving switch zone addresses in DNS file = sled-agent/src/bootstrap/early_networking.rs:264
12952024-12-13T22:28:58.116ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12962024-12-13T22:28:58.116ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12972024-12-13T22:28:58.116ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
12982024-12-13T22:28:58.116ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934 file = illumos-utils/src/zone.rs:325
12992024-12-13T22:28:58.121ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4 file = illumos-utils/src/zone.rs:325
13002024-12-13T22:28:58.127ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c file = illumos-utils/src/zone.rs:325
13012024-12-13T22:28:58.133ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919 file = illumos-utils/src/zone.rs:325
13022024-12-13T22:28:58.137ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0 file = illumos-utils/src/zone.rs:325
13032024-12-13T22:28:58.143ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5 file = illumos-utils/src/zone.rs:325
13042024-12-13T22:28:58.148ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c file = illumos-utils/src/zone.rs:325
13052024-12-13T22:28:58.297ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934 file = illumos-utils/src/zone.rs:363
13062024-12-13T22:28:58.302ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4 file = illumos-utils/src/zone.rs:363
13072024-12-13T22:28:58.308ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c file = illumos-utils/src/zone.rs:363
13082024-12-13T22:28:58.313ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919 file = illumos-utils/src/zone.rs:363
13092024-12-13T22:28:58.318ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0 file = illumos-utils/src/zone.rs:363
13102024-12-13T22:28:58.323ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5 file = illumos-utils/src/zone.rs:363
13112024-12-13T22:28:58.328ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c file = illumos-utils/src/zone.rs:363
13122024-12-13T22:28:58.333ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722 file = illumos-utils/src/zone.rs:325
13132024-12-13T22:28:58.338ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2 file = illumos-utils/src/zone.rs:325
13142024-12-13T22:28:58.343ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5 file = illumos-utils/src/zone.rs:325
13152024-12-13T22:28:58.348ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4 file = illumos-utils/src/zone.rs:325
13162024-12-13T22:28:58.353ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9 file = illumos-utils/src/zone.rs:325
13172024-12-13T22:28:58.359ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869 file = illumos-utils/src/zone.rs:325
13182024-12-13T22:28:58.365ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab file = illumos-utils/src/zone.rs:325
13192024-12-13T22:28:58.433ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
13202024-12-13T22:28:58.434ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722 file = illumos-utils/src/zone.rs:363
13212024-12-13T22:28:58.471ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2 file = illumos-utils/src/zone.rs:363
13222024-12-13T22:28:58.493ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5 file = illumos-utils/src/zone.rs:363
13232024-12-13T22:28:58.509ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4 file = illumos-utils/src/zone.rs:363
13242024-12-13T22:28:58.531ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9 file = illumos-utils/src/zone.rs:363
13252024-12-13T22:28:58.549ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869 file = illumos-utils/src/zone.rs:363
13262024-12-13T22:28:58.561ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab file = illumos-utils/src/zone.rs:363
13272024-12-13T22:28:58.637ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
13282024-12-13T22:28:58.695ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
13292024-12-13T22:28:58.750ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
13302024-12-13T22:28:58.805ZINFOSledAgent (ServiceManager): Querying MGS to determine switch location addr = fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:281
13312024-12-13T22:28:58.811ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
13322024-12-13T22:28:58.812ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
13332024-12-13T22:28:58.864ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte1", ip: 172.30.2.6, mac: MacAddr6([168, 64, 37, 255, 166, 50]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
13342024-12-13T22:28:58.864ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1263 zone_type = nexus
13352024-12-13T22:28:58.866ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
13362024-12-13T22:28:58.866ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
13372024-12-13T22:28:58.917ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte2", ip: 172.30.2.7, mac: MacAddr6([168, 64, 37, 255, 177, 163]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
13382024-12-13T22:28:58.917ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1263 zone_type = nexus
13392024-12-13T22:28:58.919ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
13402024-12-13T22:28:58.919ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
13412024-12-13T22:28:58.973ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte3", ip: 172.30.1.5, mac: MacAddr6([168, 64, 37, 255, 167, 156]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
13422024-12-13T22:28:58.973ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1263 zone_type = external_dns
13432024-12-13T22:28:58.974ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
13442024-12-13T22:28:58.974ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
13452024-12-13T22:28:59.027ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte4", ip: 172.30.2.5, mac: MacAddr6([168, 64, 37, 255, 223, 94]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.2.1 } }) }
13462024-12-13T22:28:59.027ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1263 zone_type = nexus
13472024-12-13T22:28:59.030ZINFOSledAgent (ServiceManager): Found Switch0 at fd00:1122:3344:101::2 file = sled-agent/src/bootstrap/early_networking.rs:306
13482024-12-13T22:28:59.030ZINFOSledAgent (ServiceManager): Successfully looked up all expected switch zone underlay addresses addrs = {Switch0: fd00:1122:3344:101::2} file = sled-agent/src/bootstrap/early_networking.rs:198
13492024-12-13T22:28:59.084ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte5", ip: 172.30.1.6, mac: MacAddr6([168, 64, 37, 255, 198, 39]), slot: 0, vni: Vni { inner: 100 }, subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.1.1 } }) }
13502024-12-13T22:28:59.084ZINFOSledAgent (ServiceManager): creating NAT entry for service file = sled-agent/src/services.rs:1263 zone_type = external_dns
13512024-12-13T22:28:59.102ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:83
13522024-12-13T22:28:59.102ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:90
13532024-12-13T22:28:59.109ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:83
13542024-12-13T22:28:59.109ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:90
13552024-12-13T22:28:59.109ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:83
13562024-12-13T22:28:59.109ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:90
13572024-12-13T22:28:59.109ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:83
13582024-12-13T22:28:59.109ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:90
13592024-12-13T22:28:59.109ZINFOSledAgent (ServiceManager): no nat entry found for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:83
13602024-12-13T22:28:59.109ZINFOSledAgent (ServiceManager): creating nat entry for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:90
13612024-12-13T22:28:59.129ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.165 file = clients/dpd-client/src/lib.rs:118
13622024-12-13T22:28:59.131ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
13632024-12-13T22:28:59.172ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.167 file = clients/dpd-client/src/lib.rs:118
13642024-12-13T22:28:59.175ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.168 file = clients/dpd-client/src/lib.rs:118
13652024-12-13T22:28:59.176ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
13662024-12-13T22:28:59.179ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.164 file = clients/dpd-client/src/lib.rs:118
13672024-12-13T22:28:59.179ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
13682024-12-13T22:28:59.182ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
13692024-12-13T22:28:59.182ZINFOSledAgent (ServiceManager): creation of nat entry successful for: 10.151.2.166 file = clients/dpd-client/src/lib.rs:118
13702024-12-13T22:28:59.185ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
13712024-12-13T22:28:59.349ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a file = illumos-utils/src/zone.rs:325
13722024-12-13T22:28:59.361ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93 file = illumos-utils/src/zone.rs:325
13732024-12-13T22:28:59.372ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981 file = illumos-utils/src/zone.rs:325
13742024-12-13T22:28:59.388ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae file = illumos-utils/src/zone.rs:325
13752024-12-13T22:28:59.400ZINFOSledAgent (ServiceManager): Configuring new Omicron zone: oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80 file = illumos-utils/src/zone.rs:325
13762024-12-13T22:28:59.413ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a file = illumos-utils/src/zone.rs:363
13772024-12-13T22:28:59.425ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93 file = illumos-utils/src/zone.rs:363
13782024-12-13T22:28:59.436ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981 file = illumos-utils/src/zone.rs:363
13792024-12-13T22:28:59.445ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae file = illumos-utils/src/zone.rs:363
13802024-12-13T22:28:59.454ZINFOSledAgent (ServiceManager): Installing Omicron zone: oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80 file = illumos-utils/src/zone.rs:363
13812024-12-13T22:29:08.698ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService22'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::10'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::10'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13822024-12-13T22:29:08.699ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService16'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::f'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::f'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13832024-12-13T22:29:08.699ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
13842024-12-13T22:29:08.711ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService10'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::11'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/pantry"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::11'/> <propval type="astring" name="listen_port" value='17000'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13852024-12-13T22:29:08.712ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
13862024-12-13T22:29:08.720ZINFOSledAgent (ServiceManager): Booting oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934
13872024-12-13T22:29:08.794ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService12'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::17'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_5bc1d750-0ca6-4a08-bf34-a01bf34cccef/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::17'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='8e9f7ad4-9287-434f-bef3-914833dc9296'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13882024-12-13T22:29:08.794ZINFOSledAgent (ServiceManager): Booting oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
13892024-12-13T22:29:08.803ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService13'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::19'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_35c96663-c574-4d07-a692-3eb8a2a79302/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::19'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='5f448571-c7ff-43e1-9496-a69d96bcd8c4'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13902024-12-13T22:29:08.804ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService14'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::12'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::12'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='e1275d70-5937-4dd5-be26-54e011f57dff'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13912024-12-13T22:29:08.804ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService15'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::1a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_ffd405c4-ad68-4401-9d01-3a9d70b7775f/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1a'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='1b0dc12a-e665-4efd-8519-e19eb0d02d65'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13922024-12-13T22:29:08.804ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService17'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::15'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_8159e296-b0fc-4958-9b6f-9386d6e003fa/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::15'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='cd64ca9e-6fc9-4197-80ce-4a676bbd373a'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13932024-12-13T22:29:08.805ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService19'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::16'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_fb594314-d689-4df6-8176-e09675ab8f03/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::16'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='c1698dcc-a978-4c76-af05-e1496152db64'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13942024-12-13T22:29:08.805ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService20'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::18'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_b80657b8-dd44-496f-8b60-edeffe6ea0c4/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::18'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='648530b7-87d5-4804-8c26-f8965edb79f2'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13952024-12-13T22:29:08.805ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService23'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::13'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::13'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='e5d4d3dd-d0c2-4806-9899-5aa66cd80ae8'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13962024-12-13T22:29:08.806ZINFOSledAgent (ServiceManager): Profile for oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService11'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::14'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/crucible/agent"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="dataset" value='oxp_9e30dd3c-3e72-4b01-8f5f-662db99dc37a/crucible'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::14'/> <propval type="astring" name="listen_port" value='32345'/> <propval type="astring" name="uuid" value='5d3eaf78-08c6-43de-851d-e9dc5cd51a2d'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
13972024-12-13T22:29:08.806ZINFOSledAgent (ServiceManager): Booting oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
13982024-12-13T22:29:08.814ZINFOSledAgent (ServiceManager): Booting oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
13992024-12-13T22:29:08.821ZINFOSledAgent (ServiceManager): Booting oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
14002024-12-13T22:29:08.829ZINFOSledAgent (ServiceManager): Booting oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
14012024-12-13T22:29:08.837ZINFOSledAgent (ServiceManager): Booting oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
14022024-12-13T22:29:08.845ZINFOSledAgent (ServiceManager): Booting oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4
14032024-12-13T22:29:08.852ZINFOSledAgent (ServiceManager): Booting oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
14042024-12-13T22:29:08.861ZINFOSledAgent (ServiceManager): Booting oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
14052024-12-13T22:29:09.025ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 27.553871ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
14062024-12-13T22:29:09.037ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934") failed: Property not found. retry in 73.635227ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934
14072024-12-13T22:29:09.063ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 63.140417ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
14082024-12-13T22:29:09.075ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919") failed: Property not found. retry in 48.436429ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
14092024-12-13T22:29:09.086ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 61.036999ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
14102024-12-13T22:29:09.102ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 106.284674ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
14112024-12-13T22:29:09.235ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 44.882121ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
14122024-12-13T22:29:09.431ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 46.96333ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
14132024-12-13T22:29:10.094ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 59.817112ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
14142024-12-13T22:29:11.094ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934") failed: Property not found. retry in 132.33676ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934
14152024-12-13T22:29:11.119ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4") failed: Property not found. retry in 67.425077ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
14162024-12-13T22:29:11.139ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 94.573436ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
14172024-12-13T22:29:11.153ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919") failed: Property not found. retry in 96.42878ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
14182024-12-13T22:29:11.199ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 139.311063ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
14192024-12-13T22:29:11.225ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 144.755943ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
14202024-12-13T22:29:11.243ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 149.18724ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
14212024-12-13T22:29:11.257ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 67.849411ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
14222024-12-13T22:29:11.270ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 126.639528ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
14232024-12-13T22:29:11.284ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4") failed: Property not found. retry in 64.198072ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4
14242024-12-13T22:29:11.299ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 58.581053ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
14252024-12-13T22:29:11.312ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab") failed: Property not found. retry in 41.72356ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
14262024-12-13T22:29:11.330ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934") failed: Property not found. retry in 294.80767ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934
14272024-12-13T22:29:11.343ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4") failed: Property not found. retry in 136.837567ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
14282024-12-13T22:29:11.356ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 266.79224ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
14292024-12-13T22:29:11.370ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919") failed: Property not found. retry in 225.981607ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
14302024-12-13T22:29:11.383ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 289.805622ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
14312024-12-13T22:29:11.398ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 122.794482ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
14322024-12-13T22:29:11.416ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 445.580211ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
14332024-12-13T22:29:11.432ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 67.279894ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
14342024-12-13T22:29:11.451ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 180.218302ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
14352024-12-13T22:29:11.468ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4") failed: Property not found. retry in 106.695839ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4
14362024-12-13T22:29:11.483ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 184.114889ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
14372024-12-13T22:29:11.500ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab") failed: Property not found. retry in 97.006419ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
14382024-12-13T22:29:11.516ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4") failed: Property not found. retry in 188.721218ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
14392024-12-13T22:29:11.532ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 247.327476ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
14402024-12-13T22:29:11.545ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 355.362127ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
14412024-12-13T22:29:11.597ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4") failed: Property not found. retry in 240.88806ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4
14422024-12-13T22:29:11.613ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919") failed: Property not found. retry in 560.632111ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
14432024-12-13T22:29:11.629ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab") failed: Property not found. retry in 149.709149ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
14442024-12-13T22:29:11.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934") failed: Property not found. retry in 249.23188ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934
14452024-12-13T22:29:11.662ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 231.588706ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
14462024-12-13T22:29:11.682ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 581.515388ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
14472024-12-13T22:29:11.697ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 477.168555ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
14482024-12-13T22:29:11.710ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 207.872333ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
14492024-12-13T22:29:11.725ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4") failed: Property not found. retry in 418.999083ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
14502024-12-13T22:29:11.799ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 367.314076ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
14512024-12-13T22:29:11.819ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab") failed: Property not found. retry in 410.168102ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
14522024-12-13T22:29:11.857ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4") failed: Property not found. retry in 572.208213ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4
14532024-12-13T22:29:11.879ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 1.05815341s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
14542024-12-13T22:29:11.915ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 580.721946ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
14552024-12-13T22:29:11.941ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934") failed: Property not found. retry in 589.240299ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934
14562024-12-13T22:29:11.960ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 953.218389ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
14572024-12-13T22:29:11.978ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 937.517621ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
14582024-12-13T22:29:12.162ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4") failed: Property not found. retry in 1.072069103s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
14592024-12-13T22:29:12.185ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 486.532519ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
14602024-12-13T22:29:12.202ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919") failed: Property not found. retry in 1.105826452s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
14612024-12-13T22:29:12.218ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 770.477964ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
14622024-12-13T22:29:12.246ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab") failed: Property not found. retry in 513.886431ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
14632024-12-13T22:29:12.280ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 638.74068ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
14642024-12-13T22:29:12.428ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 2.608300656s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
14652024-12-13T22:29:12.449ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4") failed: Property not found. retry in 620.760428ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4
14662024-12-13T22:29:12.511ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 1.060847844s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
14672024-12-13T22:29:12.513ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService24'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::9'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte5'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::9]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.6:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14682024-12-13T22:29:12.513ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
14692024-12-13T22:29:12.547ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934") failed: Property not found. retry in 1.04320522s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934
14702024-12-13T22:29:12.685ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 852.723504ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
14712024-12-13T22:29:12.763ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 71.973599ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
14722024-12-13T22:29:12.776ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab") failed: Property not found. retry in 1.196304744s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
14732024-12-13T22:29:12.847ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 115.88862ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
14742024-12-13T22:29:12.927ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 894.068942ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
14752024-12-13T22:29:12.937ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 1.214532701s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
14762024-12-13T22:29:12.950ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 1.272144916s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
14772024-12-13T22:29:12.968ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 1.441391476s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
14782024-12-13T22:29:12.986ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 184.461844ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
14792024-12-13T22:29:13.006ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 1.356174759s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
14802024-12-13T22:29:13.088ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4") failed: Property not found. retry in 763.277041ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4
14812024-12-13T22:29:13.202ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 584.464238ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
14822024-12-13T22:29:13.262ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4") failed: Property not found. retry in 1.488989421s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
14832024-12-13T22:29:13.327ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919") failed: Property not found. retry in 1.301685703s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
14842024-12-13T22:29:13.367ZINFOSledAgent (ServiceManager): Profile for oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService21'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::d'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/oximeter"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="id" value='689eb899-96a8-4e42-a5ca-864d1c68fdc9'/> <propval type="astring" name="address" value='[fd00:1122:3344:101::d]:12223'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14852024-12-13T22:29:13.367ZINFOSledAgent (ServiceManager): Booting oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
14862024-12-13T22:29:13.555ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 1.26478034s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
14872024-12-13T22:29:13.588ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 1.171773803s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
14882024-12-13T22:29:13.608ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934") failed: Property not found. retry in 1.105975462s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934
14892024-12-13T22:29:13.681ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9") failed: Property not found. retry in 31.473857ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
14902024-12-13T22:29:13.724ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9") failed: Property not found. retry in 109.426327ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
14912024-12-13T22:29:13.799ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 439.53176ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
14922024-12-13T22:29:13.835ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 502.105511ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
14932024-12-13T22:29:13.847ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9") failed: Property not found. retry in 111.009253ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
14942024-12-13T22:29:13.864ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4") failed: Property not found. retry in 943.504701ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4
14952024-12-13T22:29:13.875ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService26'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::c'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte2'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.7'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14962024-12-13T22:29:13.875ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService25'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::b'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte1'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.6'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
14972024-12-13T22:29:13.875ZINFOSledAgent (ServiceManager): Booting oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
14982024-12-13T22:29:13.887ZINFOSledAgent (ServiceManager): Booting oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
14992024-12-13T22:29:13.993ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9") failed: Property not found. retry in 426.247013ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
15002024-12-13T22:29:13.995ZINFOSledAgent (ServiceManager): Profile for oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService27'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::8'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte3'/> <propval type="astring" name="gateway" value='172.30.1.1'/> <propval type="astring" name="ip" value='172.30.1.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/external_dns"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::8]:5353'/> <propval type="astring" name="dns_address" value='172.30.1.5:53'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
15012024-12-13T22:29:14.019ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab") failed: Property not found. retry in 1.365378858s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
15022024-12-13T22:29:14.019ZINFOSledAgent (ServiceManager): Booting oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
15032024-12-13T22:29:14.145ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 66.302978ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
15042024-12-13T22:29:14.164ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 539.706858ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
15052024-12-13T22:29:14.198ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93") failed: Property not found. retry in 36.184848ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
15062024-12-13T22:29:14.227ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 90.65336ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
15072024-12-13T22:29:14.261ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 611.885614ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
15082024-12-13T22:29:14.502ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 1.178871578s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
15092024-12-13T22:29:14.514ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93") failed: Property not found. retry in 54.813003ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
15102024-12-13T22:29:14.528ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 1.321794526s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
15112024-12-13T22:29:14.540ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 225.724462ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
15122024-12-13T22:29:14.553ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 1.417972126s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
15132024-12-13T22:29:14.565ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae") failed: Property not found. retry in 40.97392ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
15142024-12-13T22:29:14.580ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9") failed: Property not found. retry in 518.586267ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
15152024-12-13T22:29:14.593ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 1.329436736s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
15162024-12-13T22:29:14.609ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93") failed: Property not found. retry in 260.261205ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
15172024-12-13T22:29:14.619ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae") failed: Property not found. retry in 128.110837ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
15182024-12-13T22:29:14.644ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919") failed: Property not found. retry in 677.037534ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
15192024-12-13T22:29:14.725ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 1.177975566s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
15202024-12-13T22:29:14.739ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934") failed: Property not found. retry in 604.670135ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934
15212024-12-13T22:29:14.762ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae") failed: Property not found. retry in 200.772955ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
15222024-12-13T22:29:14.782ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4") failed: Property not found. retry in 1.307999561s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
15232024-12-13T22:29:14.793ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 1.416006691s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
15242024-12-13T22:29:14.805ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 467.052604ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
15252024-12-13T22:29:14.822ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4") failed: Property not found. retry in 712.059263ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4
15262024-12-13T22:29:14.834ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 595.936203ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
15272024-12-13T22:29:14.887ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93") failed: Property not found. retry in 541.437032ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
15282024-12-13T22:29:14.901ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 1.181491341s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
15292024-12-13T22:29:14.918ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:54848
15302024-12-13T22:29:14.945ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7129 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:54848 req_id = 70bc81de-c634-4c8e-8161-e10d7eff5a72 response_code = 200 uri = /baseboard
15312024-12-13T22:29:14.976ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae") failed: Property not found. retry in 421.013496ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
15322024-12-13T22:29:15.122ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9") failed: Property not found. retry in 811.516707ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
15332024-12-13T22:29:15.144ZINFOSledAgent (ServiceManager): Profile for oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService28'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::a'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/opte-interface-setup"> <property_group type="application" name="config"> <propval type="astring" name="interface" value='opte4'/> <propval type="astring" name="gateway" value='172.30.2.1'/> <propval type="astring" name="ip" value='172.30.2.5'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/nexus"> <instance enabled="true" name="default"> <property_group type="application" name="config"> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="false" name="default"> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
15342024-12-13T22:29:15.145ZINFOSledAgent (ServiceManager): Booting oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
15352024-12-13T22:29:15.290ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 537.302013ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
15362024-12-13T22:29:15.340ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919") failed: Property not found. retry in 1.330171721s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
15372024-12-13T22:29:15.363ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934") failed: Property not found. retry in 1.025378201s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934
15382024-12-13T22:29:15.401ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab") failed: Property not found. retry in 1.358101315s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
15392024-12-13T22:29:15.422ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae") failed: Property not found. retry in 1.134238915s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
15402024-12-13T22:29:15.448ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93") failed: Property not found. retry in 861.463567ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
15412024-12-13T22:29:15.467ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 823.081278ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
15422024-12-13T22:29:15.486ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80") failed: Property not found. retry in 34.275566ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
15432024-12-13T22:29:15.538ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80") failed: Property not found. retry in 133.805527ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
15442024-12-13T22:29:15.557ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4") failed: Property not found. retry in 1.347746868s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4
15452024-12-13T22:29:15.743ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80") failed: Property not found. retry in 156.422708ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
15462024-12-13T22:29:15.759ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 538.326844ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
15472024-12-13T22:29:15.843ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 780.520725ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
15482024-12-13T22:29:15.864ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 1.010798921s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
15492024-12-13T22:29:15.925ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80") failed: Property not found. retry in 459.569731ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
15502024-12-13T22:29:15.947ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 1.317345832s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
15512024-12-13T22:29:15.967ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 1.388245754s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
15522024-12-13T22:29:15.992ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9") failed: Property not found. retry in 1.433037288s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
15532024-12-13T22:29:16.014ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 1.471186358s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
15542024-12-13T22:29:16.104ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 1.110144073s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
15552024-12-13T22:29:16.132ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4") failed: Property not found. retry in 1.070784821s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
15562024-12-13T22:29:16.242ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 1.086476363s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
15572024-12-13T22:29:16.321ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 1.468840214s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
15582024-12-13T22:29:16.343ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 591.641827ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
15592024-12-13T22:29:16.361ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93") failed: Property not found. retry in 751.560368ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
15602024-12-13T22:29:16.406ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80") failed: Property not found. retry in 1.011788093s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
15612024-12-13T22:29:16.436ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934") failed: Property not found. retry in 1.446271866s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934
15622024-12-13T22:29:16.577ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae") failed: Property not found. retry in 820.021659ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
15632024-12-13T22:29:16.650ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 1.23268379s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
15642024-12-13T22:29:16.694ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919") failed: Property not found. retry in 1.285417515s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
15652024-12-13T22:29:16.779ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab") failed: Property not found. retry in 1.433125286s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
15662024-12-13T22:29:16.894ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 850.635354ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
15672024-12-13T22:29:16.922ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4") failed: Property not found. retry in 1.30148983s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4
15682024-12-13T22:29:16.948ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 1.088313452s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
15692024-12-13T22:29:17.128ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93") failed: Property not found. retry in 924.408819ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
15702024-12-13T22:29:17.218ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4") failed: Property not found. retry in 1.152015937s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
15712024-12-13T22:29:17.233ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 1.142256288s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
15722024-12-13T22:29:17.281ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 753.800036ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
15732024-12-13T22:29:17.345ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 1.03905657s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
15742024-12-13T22:29:17.371ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 810.162721ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
15752024-12-13T22:29:17.411ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae") failed: Property not found. retry in 1.411736137s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
15762024-12-13T22:29:17.435ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80") failed: Property not found. retry in 749.594494ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
15772024-12-13T22:29:17.447ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9") failed: Property not found. retry in 1.413868153s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
15782024-12-13T22:29:17.502ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 694.735619ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
15792024-12-13T22:29:17.759ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 542.490201ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
15802024-12-13T22:29:17.804ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 1.485458244s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
15812024-12-13T22:29:17.898ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934") failed: Property not found. retry in 1.379286094s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934
15822024-12-13T22:29:17.911ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 939.40878ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
15832024-12-13T22:29:17.995ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919") failed: Property not found. retry in 1.486208298s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
15842024-12-13T22:29:18.050ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 1.319280125s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
15852024-12-13T22:29:18.066ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 535.66403ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
15862024-12-13T22:29:18.081ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93") failed: Property not found. retry in 911.560731ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
15872024-12-13T22:29:18.196ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 808.888819ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
15882024-12-13T22:29:18.210ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80") failed: Property not found. retry in 1.037762819s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
15892024-12-13T22:29:18.226ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 1.08575941s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
15902024-12-13T22:29:18.239ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4") failed: Property not found. retry in 575.494305ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4
15912024-12-13T22:29:18.253ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab") failed: Property not found. retry in 1.460440956s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
15922024-12-13T22:29:18.316ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 1.352514719s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
15932024-12-13T22:29:18.386ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4") failed: Property not found. retry in 1.204358912s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
15942024-12-13T22:29:18.399ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 1.300204614s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
15952024-12-13T22:29:18.412ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 1.365485877s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
15962024-12-13T22:29:18.615ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 627.720814ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
15972024-12-13T22:29:18.829ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4") failed: Property not found. retry in 1.461240501s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4
15982024-12-13T22:29:18.842ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae") failed: Property not found. retry in 520.979ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
15992024-12-13T22:29:18.865ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 1.053108997s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
16002024-12-13T22:29:18.880ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9") failed: Property not found. retry in 510.977692ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
16012024-12-13T22:29:19.007ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93") failed: Property not found. retry in 1.195460449s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
16022024-12-13T22:29:19.019ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 946.990833ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
16032024-12-13T22:29:19.097ZINFOSledAgent (ServiceManager): Profile for oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="oxide/zone-network-setup"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlService18'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="static_addr" value='fd00:1122:3344:101::e'/> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/ssh"> <instance enabled="false" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::e'/> <propval type="astring" name="listen_port" value='8123'/> <propval type="astring" name="store" value='/data'/> </property_group> </instance> </service> <service version="1" type="service" name="network/dns/install"> <property_group type="application" name="install_props"> <property type="net_address" name="nameserver"> <net_address_list> <value_node value="fd00:1122:3344:1::1" /> <value_node value="fd00:1122:3344:2::1" /> <value_node value="fd00:1122:3344:3::1" /> </net_address_list> </property> </property_group> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="network/dns/client"> <instance enabled="true" name="default"> </instance> </service> <service version="1" type="service" name="oxide/clickhouse-admin-single"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="http_address" value='[fd00:1122:3344:101::e]:8888'/> <propval type="astring" name="ch_address" value='[fd00:1122:3344:101::e]:8123'/> <propval type="astring" name="ch_binary" value='/opt/oxide/clickhouse/clickhouse'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34
16042024-12-13T22:29:19.097ZINFOSledAgent (ServiceManager): Booting oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2 zone file = illumos-utils/src/running_zone.rs:488 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
16052024-12-13T22:29:19.265ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 775.275812ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
16062024-12-13T22:29:19.278ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80") failed: Property not found. retry in 1.355509411s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
16072024-12-13T22:29:19.292ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934") failed: Property not found. retry in 1.284040572s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934
16082024-12-13T22:29:19.304ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 1.290066635s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
16092024-12-13T22:29:19.328ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 897.101165ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
16102024-12-13T22:29:19.377ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae") failed: Property not found. retry in 1.054354985s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
16112024-12-13T22:29:19.396ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 1.145446246s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
16122024-12-13T22:29:19.406ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 70.086303ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
16132024-12-13T22:29:19.418ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9") failed: Property not found. retry in 851.101408ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
16142024-12-13T22:29:19.487ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 65.914058ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
16152024-12-13T22:29:19.500ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919") failed: Property not found. retry in 963.086355ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
16162024-12-13T22:29:19.620ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 208.297974ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
16172024-12-13T22:29:19.651ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4") failed: Property not found. retry in 1.137557354s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
16182024-12-13T22:29:19.733ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 1.18257733s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
16192024-12-13T22:29:19.745ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 1.148125961s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
16202024-12-13T22:29:19.760ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab") failed: Property not found. retry in 1.143497444s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
16212024-12-13T22:29:19.794ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 805.020404ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
16222024-12-13T22:29:19.844ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 310.995237ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
16232024-12-13T22:29:19.934ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 1.003998129s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
16242024-12-13T22:29:19.981ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 829.072702ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
16252024-12-13T22:29:20.056ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 823.27999ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
16262024-12-13T22:29:20.174ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 520.132008ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
16272024-12-13T22:29:20.217ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93") failed: Property not found. retry in 993.492644ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
16282024-12-13T22:29:20.242ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 663.322344ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
16292024-12-13T22:29:20.282ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9") failed: Property not found. retry in 587.765275ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
16302024-12-13T22:29:20.309ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4") failed: Property not found. retry in 774.268055ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4
16312024-12-13T22:29:20.444ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae") failed: Property not found. retry in 1.345359165s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
16322024-12-13T22:29:20.475ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919") failed: Property not found. retry in 1.121772956s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
16332024-12-13T22:29:20.555ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 1.276444465s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
16342024-12-13T22:29:20.588ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934") failed: Property not found. retry in 1.41843503s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934
16352024-12-13T22:29:20.607ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 1.029584437s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
16362024-12-13T22:29:20.622ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 538.767543ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
16372024-12-13T22:29:20.648ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80") failed: Property not found. retry in 610.211495ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
16382024-12-13T22:29:20.709ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 1.371930404s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
16392024-12-13T22:29:20.802ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4") failed: Property not found. retry in 938.824518ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
16402024-12-13T22:29:20.822ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 1.041760057s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
16412024-12-13T22:29:20.884ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9") failed: Property not found. retry in 1.261399979s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
16422024-12-13T22:29:20.895ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 733.717173ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
16432024-12-13T22:29:20.906ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 1.09902999s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
16442024-12-13T22:29:20.917ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 1.060250975s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
16452024-12-13T22:29:20.928ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab") failed: Property not found. retry in 1.450947106s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
16462024-12-13T22:29:20.940ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 680.632212ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
16472024-12-13T22:29:20.950ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 1.322835195s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
16482024-12-13T22:29:21.097ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4") failed: Property not found. retry in 646.064745ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4
16492024-12-13T22:29:21.174ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 854.516007ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
16502024-12-13T22:29:21.225ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93") failed: Property not found. retry in 1.43241s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
16512024-12-13T22:29:21.272ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80") failed: Property not found. retry in 909.484322ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
16522024-12-13T22:29:21.612ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919") failed: Property not found. retry in 876.452823ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
16532024-12-13T22:29:21.634ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 624.387775ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
16542024-12-13T22:29:21.646ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 1.277873093s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
16552024-12-13T22:29:21.659ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 968.893104ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
16562024-12-13T22:29:21.751ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4") failed: Property not found. retry in 1.399929186s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
16572024-12-13T22:29:21.810ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae") failed: Property not found. retry in 798.582303ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
16582024-12-13T22:29:21.843ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 889.259651ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
16592024-12-13T22:29:21.875ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 1.096794456s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
16602024-12-13T22:29:21.988ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 872.399371ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
16612024-12-13T22:29:22.015ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 565.415056ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
16622024-12-13T22:29:22.024ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934") failed: Property not found. retry in 1.196773238s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934
16632024-12-13T22:29:22.038ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 1.174070022s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
16642024-12-13T22:29:22.091ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 535.772512ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
16652024-12-13T22:29:22.157ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9") failed: Property not found. retry in 1.154854987s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
16662024-12-13T22:29:22.191ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80") failed: Property not found. retry in 1.317042902s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
16672024-12-13T22:29:22.268ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 625.804072ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
16682024-12-13T22:29:22.283ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 1.241666991s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
16692024-12-13T22:29:22.389ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab") failed: Property not found. retry in 586.759255ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
16702024-12-13T22:29:22.501ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919") failed: Property not found. retry in 1.193363619s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
16712024-12-13T22:29:22.592ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 750.189788ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
16722024-12-13T22:29:22.618ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae") failed: Property not found. retry in 782.199048ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
16732024-12-13T22:29:22.638ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 1.255999506s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
16742024-12-13T22:29:22.646ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 1.210905676s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
16752024-12-13T22:29:22.666ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93") failed: Property not found. retry in 644.13444ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
16762024-12-13T22:29:22.742ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 821.806167ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
16772024-12-13T22:29:22.871ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 561.441848ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
16782024-12-13T22:29:22.904ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 767.309904ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
16792024-12-13T22:29:22.936ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 888.114762ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
16802024-12-13T22:29:22.982ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 680.148211ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
16812024-12-13T22:29:22.991ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab") failed: Property not found. retry in 1.228637318s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
16822024-12-13T22:29:23.161ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4") failed: Property not found. retry in 1.357606823s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
16832024-12-13T22:29:23.222ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 808.653528ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
16842024-12-13T22:29:23.320ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93") failed: Property not found. retry in 909.116439ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
16852024-12-13T22:29:23.329ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9") failed: Property not found. retry in 922.021366ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
16862024-12-13T22:29:23.351ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 780.231729ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
16872024-12-13T22:29:23.409ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae") failed: Property not found. retry in 905.432363ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
16882024-12-13T22:29:23.442ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 716.608842ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
16892024-12-13T22:29:23.518ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80") failed: Property not found. retry in 979.688375ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
16902024-12-13T22:29:23.534ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 1.175057949s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
16912024-12-13T22:29:23.572ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 623.143946ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
16922024-12-13T22:29:23.671ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 1.052903735s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
16932024-12-13T22:29:23.681ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 939.311364ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
16942024-12-13T22:29:23.703ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919") failed: Property not found. retry in 1.414260346s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
16952024-12-13T22:29:23.833ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 1.489180882s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
16962024-12-13T22:29:23.867ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 875.524758ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
16972024-12-13T22:29:23.904ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 1.228987272s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
16982024-12-13T22:29:24.042ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 1.02891603s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
16992024-12-13T22:29:24.142ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 1.218833223s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
17002024-12-13T22:29:24.167ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 1.157495951s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
17012024-12-13T22:29:24.204ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 1.150286422s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
17022024-12-13T22:29:24.228ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab") failed: Property not found. retry in 1.383406775s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
17032024-12-13T22:29:24.238ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93") failed: Property not found. retry in 643.650769ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
17042024-12-13T22:29:24.261ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9") failed: Property not found. retry in 1.287798963s file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
17052024-12-13T22:29:24.323ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae") failed: Property not found. retry in 1.211010837s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
17062024-12-13T22:29:24.509ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80") failed: Property not found. retry in 578.180401ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
17072024-12-13T22:29:24.529ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4") failed: Property not found. retry in 1.11500939s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
17082024-12-13T22:29:24.630ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 699.53858ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
17092024-12-13T22:29:24.720ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 680.687206ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
17102024-12-13T22:29:24.734ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 544.729933ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
17112024-12-13T22:29:24.752ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 584.108033ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
17122024-12-13T22:29:24.892ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93") failed: Property not found. retry in 742.400046ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
17132024-12-13T22:29:25.087ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5") failed: Property not found. retry in 1.40483527s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
17142024-12-13T22:29:25.097ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80") failed: Property not found. retry in 1.136982653s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
17152024-12-13T22:29:25.127ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919") failed: Property not found. retry in 606.205665ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
17162024-12-13T22:29:25.142ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 1.29736775s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17172024-12-13T22:29:25.288ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869") failed: Property not found. retry in 1.421487799s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
17182024-12-13T22:29:25.333ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a") failed: Property not found. retry in 1.217280624s file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
17192024-12-13T22:29:25.342ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5") failed: Property not found. retry in 1.06039913s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
17202024-12-13T22:29:25.350ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 603.134727ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
17212024-12-13T22:29:25.359ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722") failed: Property not found. retry in 1.052796342s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
17222024-12-13T22:29:25.367ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 967.309927ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
17232024-12-13T22:29:25.376ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c") failed: Property not found. retry in 1.116315006s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
17242024-12-13T22:29:25.411ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 743.823623ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
17252024-12-13T22:29:25.543ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae") failed: Property not found. retry in 957.212934ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
17262024-12-13T22:29:25.560ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9") failed: Property not found. retry in 574.51073ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
17272024-12-13T22:29:25.622ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab") failed: Property not found. retry in 679.720894ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
17282024-12-13T22:29:25.645ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93") failed: Property not found. retry in 980.530597ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
17292024-12-13T22:29:25.654ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4") failed: Property not found. retry in 1.188325097s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
17302024-12-13T22:29:25.744ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919") failed: Property not found. retry in 1.420451459s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
17312024-12-13T22:29:25.966ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c") failed: Property not found. retry in 1.059452772s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
17322024-12-13T22:29:26.146ZWARNSledAgent (ServiceManager): wait for service Some("oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9") failed: Property not found. retry in 609.937424ms file = illumos-utils/src/svc.rs:36 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
17332024-12-13T22:29:26.164ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 743.334945ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
17342024-12-13T22:29:26.244ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80") failed: Property not found. retry in 897.061026ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
17352024-12-13T22:29:26.313ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab") failed: Property not found. retry in 554.733248ms file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
17362024-12-13T22:29:26.345ZWARNSledAgent (ServiceManager): wait for service Some("oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0") failed: Property not found. retry in 1.138223267s file = illumos-utils/src/svc.rs:36 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
17372024-12-13T22:29:26.449ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 1.002076015s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17382024-12-13T22:29:26.530ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae") failed: Property not found. retry in 722.420829ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
17392024-12-13T22:29:26.637ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93") failed: Property not found. retry in 1.181699512s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
17402024-12-13T22:29:26.995ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 502.173968ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
17412024-12-13T22:29:27.151ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80") failed: Property not found. retry in 761.159019ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
17422024-12-13T22:29:27.340ZWARNSledAgent (ServiceManager): wait for service Some("oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae") failed: Property not found. retry in 785.166754ms file = illumos-utils/src/svc.rs:36 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
17432024-12-13T22:29:27.461ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 1.410200412s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17442024-12-13T22:29:27.508ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981") failed: Property not found. retry in 1.44984046s file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
17452024-12-13T22:29:27.922ZWARNSledAgent (ServiceManager): wait for service Some("oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80") failed: Property not found. retry in 604.184977ms file = illumos-utils/src/svc.rs:36 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
17462024-12-13T22:29:28.882ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 1.221074573s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17472024-12-13T22:29:30.112ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 5.173473607s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
17482024-12-13T22:29:30.116ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 1.160519958s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17492024-12-13T22:29:31.286ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 1.499037861s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17502024-12-13T22:29:32.801ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 1.250207922s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17512024-12-13T22:29:34.063ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 579.265065ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17522024-12-13T22:29:34.654ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 614.266393ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17532024-12-13T22:29:35.280ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 933.480342ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17542024-12-13T22:29:35.366ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 4.259877862s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/producers)" file = oximeter/producer/src/lib.rs:420 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
17552024-12-13T22:29:36.225ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 1.230226481s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17562024-12-13T22:29:37.466ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 1.210098841s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17572024-12-13T22:29:38.688ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 1.067311819s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17582024-12-13T22:29:39.766ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 1.257153871s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17592024-12-13T22:29:41.034ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 847.492368ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17602024-12-13T22:29:41.893ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 1.41653615s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17612024-12-13T22:29:42.013ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 10.903435998s error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"535a85d1-9b83-4d26-b94c-dc94f6cc413d\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Fri, 13 Dec 2024 22:29:42 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"535a85d1-9b83-4d26-b94c-dc94f6cc413d\\" }" file = oximeter/producer/src/lib.rs:420 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
17622024-12-13T22:29:43.325ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 693.719382ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17632024-12-13T22:29:44.032ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 543.175181ms file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17642024-12-13T22:29:44.589ZWARNSledAgent (ServiceManager): wait for service Some("oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2") failed: Property not found. retry in 1.463320176s file = illumos-utils/src/svc.rs:36 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17652024-12-13T22:29:45.024ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:48878
17662024-12-13T22:29:45.025ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:48878 req_id = 71ada154-c815-42c4-86f9-874fc1dbbd76 response_code = 200 uri = /baseboard
17672024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_pantry_128c56c1-c87f-4290-bec9-772d8c30d934
17682024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_d64a1818-8633-453a-b13f-88ca4e89a7c4
17692024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_external_dns_0ef774ca-06b2-40a8-a632-7304d594c07a
17702024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_79db7e3d-21d6-43fe-bcec-aa471a9e012c
17712024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_45856571-2c50-487e-9962-2d48d24e4919
17722024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_ceccfc26-d275-4f22-bea6-d0c86043e1f0
17732024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_74b96c49-e4d9-4174-9313-533584a77cb5
17742024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_nexus_ef47bd05-ad7e-42a3-8257-dec06acd2b93
17752024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_nexus_7e3d78a7-b6c3-4636-b217-b0a79485c981
17762024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_pantry_fef903b4-5e26-41cf-9ef8-fdf744c1ce8c
17772024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_75f07622-aad2-4d27-9e07-2856e4fb4722
17782024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_clickhouse_2a7c9cd1-70fc-408c-9d4d-3dd91aa313b2
17792024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_35d09a11-5d25-47f0-a6f4-341cbb551dd5
17802024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_694b4960-fc0f-482e-8da6-0ffd414578b4
17812024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_external_dns_3e0d656e-3552-43ff-8e95-5ed82471e0ae
17822024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_oximeter_689eb899-96a8-4e42-a5ca-864d1c68fdc9
17832024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_pantry_0dd7b0ec-9c6d-4813-9a5e-cd184cdec869
17842024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_crucible_acec6218-e24c-44eb-99aa-6d30724f14ab
17852024-12-13T22:29:46.099ZINFOSledAgent (ServiceManager): Zone started file = sled-agent/src/services.rs:3310 zone = oxz_nexus_b2389da8-cde2-4174-a8e1-a4dabd361b80
17862024-12-13T22:29:46.104ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
17872024-12-13T22:29:46.116ZINFOSledAgent (ServiceManager): Writing ledger to /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/.omicron-zones.json.tmp file = common/src/ledger.rs:199
17882024-12-13T22:29:46.145ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 49377973 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::1]:48088 req_id = 03697880-69fc-4dea-955b-d6c4daa3c059 response_code = 204 uri = /omicron-zones
17892024-12-13T22:29:46.230ZINFOSledAgent (RSS): Finished setting up services file = sled-agent/src/rack_setup/service.rs:1415
17902024-12-13T22:29:46.237ZINFOSledAgent (RSS): Handing off control to Nexus file = sled-agent/src/rack_setup/service.rs:790
17912024-12-13T22:29:46.243ZINFOSledAgent (RSS): Nexus address: [fd00:1122:3344:101::c]:12221 file = sled-agent/src/rack_setup/service.rs:802
17922024-12-13T22:29:46.289ZINFOSledAgent (RSS): rack_network_config: RackNetworkConfigV2 { bfd: [], bgp: [], infra_ip_first: 10.151.2.174, infra_ip_last: 10.151.2.174, ports: [ PortConfigV2 { addresses: [ UplinkAddressConfig { address: V4( Ipv4Net { addr: 10.151.2.174, width: 24, }, ), vlan_id: None, }, ], autoneg: false, bgp_peers: [], lldp: None, port: "qsfp0", routes: [ RouteConfig { destination: V4( Ipv4Net { addr: 0.0.0.0, width: 0, }, ), nexthop: 10.151.2.100, rib_priority: None, vlan_id: None, }, ], switch: Switch0, tx_eq: None, uplink_port_fec: Some( None, ), uplink_port_speed: Speed40G, }, ], rack_subnet: Ipv6Net { addr: fd00:1122:3344:100::, width: 56, }, } file = sled-agent/src/rack_setup/service.rs:1016
17932024-12-13T22:29:47.303ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "6b03b0aa-18e2-49d9-a48f-4d4bcc2ab1bb", "content-length": "173", "date": "Fri, 13 Dec 2024 22:29:46 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"35eb16d3-adec-4806-a15c-ab7a15d18765\"", request_id: "6b03b0aa-18e2-49d9-a48f-4d4bcc2ab1bb" } file = sled-agent/src/rack_setup/service.rs:1078
17942024-12-13T22:29:47.952ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "37442869-c05b-4bb4-99eb-cc7b06a0e680", "content-length": "173", "date": "Fri, 13 Dec 2024 22:29:47 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"35eb16d3-adec-4806-a15c-ab7a15d18765\"", request_id: "37442869-c05b-4bb4-99eb-cc7b06a0e680" } file = sled-agent/src/rack_setup/service.rs:1078
17952024-12-13T22:29:48.436ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "e71cfc4b-4e74-4ec2-85d5-9e765848e5fb", "content-length": "173", "date": "Fri, 13 Dec 2024 22:29:48 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"35eb16d3-adec-4806-a15c-ab7a15d18765\"", request_id: "e71cfc4b-4e74-4ec2-85d5-9e765848e5fb" } file = sled-agent/src/rack_setup/service.rs:1078
17962024-12-13T22:29:48.755ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "3296d368-f638-417c-9509-3edf3b500729", "content-length": "173", "date": "Fri, 13 Dec 2024 22:29:48 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"35eb16d3-adec-4806-a15c-ab7a15d18765\"", request_id: "3296d368-f638-417c-9509-3edf3b500729" } file = sled-agent/src/rack_setup/service.rs:1078
17972024-12-13T22:29:49.444ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "e82374bd-3202-4df7-bb79-d0c646281bf9", "content-length": "173", "date": "Fri, 13 Dec 2024 22:29:49 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"35eb16d3-adec-4806-a15c-ab7a15d18765\"", request_id: "e82374bd-3202-4df7-bb79-d0c646281bf9" } file = sled-agent/src/rack_setup/service.rs:1078
17982024-12-13T22:29:49.964ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "34010759-5e80-4a1c-82f5-3780e8a0e102", "content-length": "173", "date": "Fri, 13 Dec 2024 22:29:49 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"35eb16d3-adec-4806-a15c-ab7a15d18765\"", request_id: "34010759-5e80-4a1c-82f5-3780e8a0e102" } file = sled-agent/src/rack_setup/service.rs:1078
17992024-12-13T22:29:50.421ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "0146208d-e46f-4888-8297-a5fff270713b", "content-length": "173", "date": "Fri, 13 Dec 2024 22:29:49 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"35eb16d3-adec-4806-a15c-ab7a15d18765\"", request_id: "0146208d-e46f-4888-8297-a5fff270713b" } file = sled-agent/src/rack_setup/service.rs:1078
18002024-12-13T22:29:51.112ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "034013e6-fce7-464e-856b-79f688b2980d", "content-length": "173", "date": "Fri, 13 Dec 2024 22:29:50 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"35eb16d3-adec-4806-a15c-ab7a15d18765\"", request_id: "034013e6-fce7-464e-856b-79f688b2980d" } file = sled-agent/src/rack_setup/service.rs:1078
18012024-12-13T22:29:51.961ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "9c1b920b-8eb0-4b63-bc3c-25a3122a93fd", "content-length": "173", "date": "Fri, 13 Dec 2024 22:29:51 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"35eb16d3-adec-4806-a15c-ab7a15d18765\"", request_id: "9c1b920b-8eb0-4b63-bc3c-25a3122a93fd" } file = sled-agent/src/rack_setup/service.rs:1078
18022024-12-13T22:29:52.524ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "a00b75df-444f-4067-8cbf-923bc723719e", "content-length": "173", "date": "Fri, 13 Dec 2024 22:29:52 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"35eb16d3-adec-4806-a15c-ab7a15d18765\"", request_id: "a00b75df-444f-4067-8cbf-923bc723719e" } file = sled-agent/src/rack_setup/service.rs:1078
18032024-12-13T22:29:52.956ZWARNSledAgent (producer-registration-task): failed to register as a producer with Nexus, will retry delay = 44.907946529s error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"a1718ec9-bd1f-41a4-97d7-defc0a62e521\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Fri, 13 Dec 2024 22:29:52 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"a1718ec9-bd1f-41a4-97d7-defc0a62e521\\" }" file = oximeter/producer/src/lib.rs:420 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
18042024-12-13T22:29:53.317ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "ffb37728-d868-4ca9-b523-62e6e3add61a", "content-length": "173", "date": "Fri, 13 Dec 2024 22:29:53 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"35eb16d3-adec-4806-a15c-ab7a15d18765\"", request_id: "ffb37728-d868-4ca9-b523-62e6e3add61a" } file = sled-agent/src/rack_setup/service.rs:1078
18052024-12-13T22:29:54.522ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "6b89ec8a-e39e-4be1-9395-3bf993b6c785", "content-length": "173", "date": "Fri, 13 Dec 2024 22:29:54 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"35eb16d3-adec-4806-a15c-ab7a15d18765\"", request_id: "6b89ec8a-e39e-4be1-9395-3bf993b6c785" } file = sled-agent/src/rack_setup/service.rs:1078
18062024-12-13T22:29:55.509ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "13066675-56ab-442e-91c1-15fa42125832", "content-length": "173", "date": "Fri, 13 Dec 2024 22:29:55 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"35eb16d3-adec-4806-a15c-ab7a15d18765\"", request_id: "13066675-56ab-442e-91c1-15fa42125832" } file = sled-agent/src/rack_setup/service.rs:1078
18072024-12-13T22:29:56.950ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "c8f8bcd0-21ce-43d4-a7da-dfca6f07c842", "content-length": "173", "date": "Fri, 13 Dec 2024 22:29:56 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"35eb16d3-adec-4806-a15c-ab7a15d18765\"", request_id: "c8f8bcd0-21ce-43d4-a7da-dfca6f07c842" } file = sled-agent/src/rack_setup/service.rs:1078
18082024-12-13T22:29:58.896ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "5291793f-e48d-427a-81ab-87e2f7316bee", "content-length": "173", "date": "Fri, 13 Dec 2024 22:29:58 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"35eb16d3-adec-4806-a15c-ab7a15d18765\"", request_id: "5291793f-e48d-427a-81ab-87e2f7316bee" } file = sled-agent/src/rack_setup/service.rs:1078
18092024-12-13T22:30:00.871ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "414bfbb9-824e-41e3-aae9-d76fd1ca48c6", "content-length": "173", "date": "Fri, 13 Dec 2024 22:30:00 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"35eb16d3-adec-4806-a15c-ab7a15d18765\"", request_id: "414bfbb9-824e-41e3-aae9-d76fd1ca48c6" } file = sled-agent/src/rack_setup/service.rs:1078
18102024-12-13T22:30:03.219ZINFOSledAgent (RSS): Failed to handoff to nexus: Error Response: status: 404 Not Found; headers: {"content-type": "application/json", "x-request-id": "ae6d64f5-70fa-4c61-920e-d48ab160a1ef", "content-length": "173", "date": "Fri, 13 Dec 2024 22:30:03 GMT"}; value: Error { error_code: Some("ObjectNotFound"), message: "not found: sled with id \"35eb16d3-adec-4806-a15c-ab7a15d18765\"", request_id: "ae6d64f5-70fa-4c61-920e-d48ab160a1ef" } file = sled-agent/src/rack_setup/service.rs:1078
18112024-12-13T22:30:04.202ZINFOSledAgent (NexusNotifierTask): Successfully put SledAgentInfo to nexus file = sled-agent/src/nexus.rs:500 new_generation = 1 old_generation = 1 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
18122024-12-13T22:30:04.294ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38671
18132024-12-13T22:30:04.298ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
18142024-12-13T22:30:04.298ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
18152024-12-13T22:30:04.299ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:177
18162024-12-13T22:30:04.300ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:177
18172024-12-13T22:30:04.315ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18182024-12-13T22:30:04.315ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5
18192024-12-13T22:30:04.343ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18202024-12-13T22:30:04.343ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f
18212024-12-13T22:30:04.368ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18222024-12-13T22:30:04.368ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_9e30dd3c-3e72-4b01-8f5f-662db99dc37a
18232024-12-13T22:30:04.393ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18242024-12-13T22:30:04.393ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8159e296-b0fc-4958-9b6f-9386d6e003fa
18252024-12-13T22:30:04.417ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18262024-12-13T22:30:04.417ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_fb594314-d689-4df6-8176-e09675ab8f03
18272024-12-13T22:30:04.440ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18282024-12-13T22:30:04.440ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_5bc1d750-0ca6-4a08-bf34-a01bf34cccef
18292024-12-13T22:30:04.464ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18302024-12-13T22:30:04.464ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_b80657b8-dd44-496f-8b60-edeffe6ea0c4
18312024-12-13T22:30:04.489ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18322024-12-13T22:30:04.489ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_35c96663-c574-4d07-a692-3eb8a2a79302
18332024-12-13T22:30:04.513ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18342024-12-13T22:30:04.513ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_ffd405c4-ad68-4401-9d01-3a9d70b7775f
18352024-12-13T22:30:04.526ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 231139 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38671 req_id = 9984445b-8622-4f86-81c1-27190e5e0766 response_code = 200 uri = /inventory
18362024-12-13T22:30:05.525ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46135
18372024-12-13T22:30:05.528ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0e0028b4-c96e-4259-b0d9-2dd1e967b57e, kind: Service { id: 3e0d656e-3552-43ff-8e95-5ed82471e0ae }, name: Name("external-dns-3e0d656e-3552-43ff-8e95-5ed82471e0ae"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 156])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9fbc6464-5658-4709-85f5-cba4ad21c9de, kind: Service { id: 0ef774ca-06b2-40a8-a632-7304d594c07a }, name: Name("external-dns-0ef774ca-06b2-40a8-a632-7304d594c07a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 39])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: bb1ce312-163f-477c-9052-27f110965501, kind: Service { id: ef47bd05-ad7e-42a3-8257-dec06acd2b93 }, name: Name("nexus-ef47bd05-ad7e-42a3-8257-dec06acd2b93"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 50])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: be9ea2e0-22c4-425c-9afb-1597ead9d56b, kind: Service { id: b2389da8-cde2-4174-a8e1-a4dabd361b80 }, name: Name("nexus-b2389da8-cde2-4174-a8e1-a4dabd361b80"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 223, 94])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e47aafa9-7219-497b-8e24-b813038ee602, kind: Service { id: 7e3d78a7-b6c3-4636-b217-b0a79485c981 }, name: Name("nexus-7e3d78a7-b6c3-4636-b217-b0a79485c981"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 163])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
18382024-12-13T22:30:05.528ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18392024-12-13T22:30:05.528ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
18402024-12-13T22:30:05.528ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18412024-12-13T22:30:05.528ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18422024-12-13T22:30:05.528ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18432024-12-13T22:30:05.528ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18442024-12-13T22:30:05.528ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2996 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:46135 req_id = 6bb93ec7-62de-47c3-b5dd-01d2c548b1bb response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18452024-12-13T22:30:05.530ZINFOSledAgent (RSS): Handoff to Nexus is complete file = sled-agent/src/rack_setup/service.rs:1088
18462024-12-13T22:30:05.533ZINFOSledAgent (RSS): Writing ledger to /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
18472024-12-13T22:30:05.534ZINFOSledAgent (RSS): Writing ledger to /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/.rss-plan-completed.marker.tmp file = common/src/ledger.rs:199
18482024-12-13T22:30:05.831ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62543
18492024-12-13T22:30:05.832ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
18502024-12-13T22:30:05.832ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
18512024-12-13T22:30:05.832ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:177
18522024-12-13T22:30:05.835ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:177
18532024-12-13T22:30:05.849ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18542024-12-13T22:30:05.849ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5
18552024-12-13T22:30:05.877ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18562024-12-13T22:30:05.877ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f
18572024-12-13T22:30:05.902ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18582024-12-13T22:30:05.902ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_9e30dd3c-3e72-4b01-8f5f-662db99dc37a
18592024-12-13T22:30:05.927ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18602024-12-13T22:30:05.927ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8159e296-b0fc-4958-9b6f-9386d6e003fa
18612024-12-13T22:30:05.937ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63753
18622024-12-13T22:30:05.937ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0e0028b4-c96e-4259-b0d9-2dd1e967b57e, kind: Service { id: 3e0d656e-3552-43ff-8e95-5ed82471e0ae }, name: Name("external-dns-3e0d656e-3552-43ff-8e95-5ed82471e0ae"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 156])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9fbc6464-5658-4709-85f5-cba4ad21c9de, kind: Service { id: 0ef774ca-06b2-40a8-a632-7304d594c07a }, name: Name("external-dns-0ef774ca-06b2-40a8-a632-7304d594c07a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 39])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: bb1ce312-163f-477c-9052-27f110965501, kind: Service { id: ef47bd05-ad7e-42a3-8257-dec06acd2b93 }, name: Name("nexus-ef47bd05-ad7e-42a3-8257-dec06acd2b93"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 50])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: be9ea2e0-22c4-425c-9afb-1597ead9d56b, kind: Service { id: b2389da8-cde2-4174-a8e1-a4dabd361b80 }, name: Name("nexus-b2389da8-cde2-4174-a8e1-a4dabd361b80"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 223, 94])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e47aafa9-7219-497b-8e24-b813038ee602, kind: Service { id: 7e3d78a7-b6c3-4636-b217-b0a79485c981 }, name: Name("nexus-7e3d78a7-b6c3-4636-b217-b0a79485c981"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 163])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
18632024-12-13T22:30:05.937ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18642024-12-13T22:30:05.937ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
18652024-12-13T22:30:05.937ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18662024-12-13T22:30:05.937ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18672024-12-13T22:30:05.937ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18682024-12-13T22:30:05.937ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18692024-12-13T22:30:05.937ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 502 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:63753 req_id = aa4171c9-8878-4a57-88e5-66e9a183176f response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18702024-12-13T22:30:05.952ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18712024-12-13T22:30:05.952ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_fb594314-d689-4df6-8176-e09675ab8f03
18722024-12-13T22:30:05.978ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18732024-12-13T22:30:05.978ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_5bc1d750-0ca6-4a08-bf34-a01bf34cccef
18742024-12-13T22:30:06.003ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18752024-12-13T22:30:06.003ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_b80657b8-dd44-496f-8b60-edeffe6ea0c4
18762024-12-13T22:30:06.030ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18772024-12-13T22:30:06.030ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_35c96663-c574-4d07-a692-3eb8a2a79302
18782024-12-13T22:30:06.052ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
18792024-12-13T22:30:06.052ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_ffd405c4-ad68-4401-9d01-3a9d70b7775f
18802024-12-13T22:30:06.067ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 235510 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62543 req_id = 7d1bbc92-9c34-4363-8c11-bcb5da7004e1 response_code = 200 uri = /inventory
18812024-12-13T22:30:06.114ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:44854
18822024-12-13T22:30:06.115ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0e0028b4-c96e-4259-b0d9-2dd1e967b57e, kind: Service { id: 3e0d656e-3552-43ff-8e95-5ed82471e0ae }, name: Name("external-dns-3e0d656e-3552-43ff-8e95-5ed82471e0ae"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 156])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9fbc6464-5658-4709-85f5-cba4ad21c9de, kind: Service { id: 0ef774ca-06b2-40a8-a632-7304d594c07a }, name: Name("external-dns-0ef774ca-06b2-40a8-a632-7304d594c07a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 39])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: bb1ce312-163f-477c-9052-27f110965501, kind: Service { id: ef47bd05-ad7e-42a3-8257-dec06acd2b93 }, name: Name("nexus-ef47bd05-ad7e-42a3-8257-dec06acd2b93"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 50])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: be9ea2e0-22c4-425c-9afb-1597ead9d56b, kind: Service { id: b2389da8-cde2-4174-a8e1-a4dabd361b80 }, name: Name("nexus-b2389da8-cde2-4174-a8e1-a4dabd361b80"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 223, 94])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e47aafa9-7219-497b-8e24-b813038ee602, kind: Service { id: 7e3d78a7-b6c3-4636-b217-b0a79485c981 }, name: Name("nexus-7e3d78a7-b6c3-4636-b217-b0a79485c981"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 163])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
18832024-12-13T22:30:06.115ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18842024-12-13T22:30:06.115ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
18852024-12-13T22:30:06.115ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18862024-12-13T22:30:06.115ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18872024-12-13T22:30:06.115ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18882024-12-13T22:30:06.115ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18892024-12-13T22:30:06.115ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 382 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:44854 req_id = a4e65328-a3ea-483f-89c8-452a09cae437 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18902024-12-13T22:30:06.863ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39209
18912024-12-13T22:30:06.863ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0e0028b4-c96e-4259-b0d9-2dd1e967b57e, kind: Service { id: 3e0d656e-3552-43ff-8e95-5ed82471e0ae }, name: Name("external-dns-3e0d656e-3552-43ff-8e95-5ed82471e0ae"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 156])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9fbc6464-5658-4709-85f5-cba4ad21c9de, kind: Service { id: 0ef774ca-06b2-40a8-a632-7304d594c07a }, name: Name("external-dns-0ef774ca-06b2-40a8-a632-7304d594c07a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 39])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: bb1ce312-163f-477c-9052-27f110965501, kind: Service { id: ef47bd05-ad7e-42a3-8257-dec06acd2b93 }, name: Name("nexus-ef47bd05-ad7e-42a3-8257-dec06acd2b93"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 50])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: be9ea2e0-22c4-425c-9afb-1597ead9d56b, kind: Service { id: b2389da8-cde2-4174-a8e1-a4dabd361b80 }, name: Name("nexus-b2389da8-cde2-4174-a8e1-a4dabd361b80"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 223, 94])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e47aafa9-7219-497b-8e24-b813038ee602, kind: Service { id: 7e3d78a7-b6c3-4636-b217-b0a79485c981 }, name: Name("nexus-7e3d78a7-b6c3-4636-b217-b0a79485c981"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 163])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
18922024-12-13T22:30:06.863ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18932024-12-13T22:30:06.863ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
18942024-12-13T22:30:06.863ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
18952024-12-13T22:30:06.863ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18962024-12-13T22:30:06.863ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18972024-12-13T22:30:06.863ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
18982024-12-13T22:30:06.863ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 248 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:39209 req_id = b59dba70-9a2a-4049-b32c-bb917804c0d0 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
18992024-12-13T22:30:13.034ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60414
19002024-12-13T22:30:13.037ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2639 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60414 req_id = 47091199-5d1d-4a70-85f6-e6de8e22126c response_code = 200 uri = /vpc-routes
19012024-12-13T22:30:13.101ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
19022024-12-13T22:30:13.101ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 821 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:60414 req_id = ede42fc6-1392-42af-92b5-56bfff77ccab response_code = 204 uri = /eip-gateways
19032024-12-13T22:30:13.231ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3065 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:60414 req_id = 58439465-7977-4ae7-9c18-a77ba4ce8d0b response_code = 204 uri = /vpc-routes
19042024-12-13T22:30:14.864ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44007
19052024-12-13T22:30:14.864ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44007 req_id = 44f29b52-a030-4c12-a5f7-9fba2baee8b0 response_code = 200 uri = /vpc-routes
19062024-12-13T22:30:14.935ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
19072024-12-13T22:30:14.935ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 85 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44007 req_id = 88c3c989-5220-4c15-9f77-66fd1c5648b6 response_code = 204 uri = /eip-gateways
19082024-12-13T22:30:14.954ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 235 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44007 req_id = 8941078f-38b7-445c-ad1f-60b3f6054550 response_code = 204 uri = /vpc-routes
19092024-12-13T22:30:15.040ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34399
19102024-12-13T22:30:15.040ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 279 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34399 req_id = 13206b9b-2a3b-4c88-99cc-633a709241e2 response_code = 200 uri = /v2p
19112024-12-13T22:30:15.063ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:36228
19122024-12-13T22:30:15.064ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 33 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:36228 req_id = 49789a47-d153-4e7f-abdd-6b7e80ea8829 response_code = 200 uri = /baseboard
19132024-12-13T22:30:15.238ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:53754
19142024-12-13T22:30:15.239ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19152024-12-13T22:30:15.310ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19162024-12-13T22:30:15.372ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 134295 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:53754 req_id = 23f19456-df50-460b-8b2f-2f9c935c591c response_code = 204 uri = /switch-ports
19172024-12-13T22:30:15.375ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2039 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:53754 req_id = 0442a2f1-fa1a-42dd-a35f-8a61b5615d98 response_code = 200 uri = /network-bootstore-config
19182024-12-13T22:30:15.426ZINFOSledAgent (bootstore): Attempting to update network config with generation: 2, current_generation: 1 file = bootstore/src/schemes/v0/peer.rs:564 peer_id = pc-sock-i86pc
19192024-12-13T22:30:15.427ZINFOSledAgent (bootstore): Writing ledger to /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
19202024-12-13T22:30:15.428ZINFOSledAgent (bootstore): Writing ledger to /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/cluster/.bootstore-network-config.json.tmp file = common/src/ledger.rs:199 peer_id = pc-sock-i86pc
19212024-12-13T22:30:15.429ZINFOSledAgent (bootstore): Broadcasting network config with generation 2 file = bootstore/src/schemes/v0/peer.rs:648 peer_id = pc-sock-i86pc
19222024-12-13T22:30:15.430ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3600 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:53754 req_id = 0c13ddf3-ce07-4286-869f-f7b7e321718d response_code = 204 uri = /network-bootstore-config
19232024-12-13T22:30:16.134ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47494
19242024-12-13T22:30:16.134ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 34 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47494 req_id = 45789af1-d89a-4a7c-a337-731bb2fcfcb4 response_code = 200 uri = /vpc-routes
19252024-12-13T22:30:16.189ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43978
19262024-12-13T22:30:16.189ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 29 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43978 req_id = 8050e882-a34d-4c3f-b6ba-a0bccdd1bfa4 response_code = 200 uri = /vpc-routes
19272024-12-13T22:30:16.195ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62385
19282024-12-13T22:30:16.195ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 121 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62385 req_id = cd3470cd-6e0a-49b2-a4be-5aebf77b912a response_code = 200 uri = /v2p
19292024-12-13T22:30:16.267ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
19302024-12-13T22:30:16.267ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47494 req_id = 98d9c4af-2ae7-4e68-be5c-f4266b04e271 response_code = 204 uri = /eip-gateways
19312024-12-13T22:30:16.277ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
19322024-12-13T22:30:16.277ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 37 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43978 req_id = 4e706864-099f-4400-9fe5-b059db2d02e5 response_code = 204 uri = /eip-gateways
19332024-12-13T22:30:16.279ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47494 req_id = 748a1d83-0265-472b-bb65-0f1a32245ebf response_code = 204 uri = /vpc-routes
19342024-12-13T22:30:16.290ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43978 req_id = 0083cbbf-a752-453a-aa43-0b6b9f9c2920 response_code = 204 uri = /vpc-routes
19352024-12-13T22:30:16.302ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53534
19362024-12-13T22:30:16.302ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19372024-12-13T22:30:16.340ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19382024-12-13T22:30:16.391ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89064 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:53534 req_id = 1d23c906-e8d5-4a5a-9989-54193ad7585f response_code = 204 uri = /switch-ports
19392024-12-13T22:30:16.391ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36177
19402024-12-13T22:30:16.391ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40275
19412024-12-13T22:30:16.392ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 227 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36177 req_id = 5cbc64cf-43f8-4597-945e-5f3e8d9f9fa7 response_code = 200 uri = /v2p
19422024-12-13T22:30:16.392ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 394 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53534 req_id = 8136062c-77cb-4c8b-b531-4ced7dd9f537 response_code = 200 uri = /network-bootstore-config
19432024-12-13T22:30:16.392ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19442024-12-13T22:30:16.430ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19452024-12-13T22:30:16.484ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92724 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:40275 req_id = 1e729940-fc34-499d-b9c6-4da9d91e4924 response_code = 204 uri = /switch-ports
19462024-12-13T22:30:16.485ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40275 req_id = 1d795634-135e-4dcf-ac7e-a44f49542938 response_code = 200 uri = /network-bootstore-config
19472024-12-13T22:30:45.100ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:52788
19482024-12-13T22:30:45.100ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:52788 req_id = dac98f3e-db27-447c-9c9b-8f37521e2d41 response_code = 200 uri = /baseboard
19492024-12-13T22:30:45.291ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:42582
19502024-12-13T22:30:45.291ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:42582 req_id = a5c12f68-a813-4c81-9d0f-9cfd9ef3c7d2 response_code = 200 uri = /vpc-routes
19512024-12-13T22:30:45.312ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57936
19522024-12-13T22:30:45.312ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 175 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57936 req_id = 64713e0e-92d6-426e-be61-e9ebd371eee6 response_code = 200 uri = /v2p
19532024-12-13T22:30:45.340ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
19542024-12-13T22:30:45.340ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 73 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42582 req_id = 3fbd5da3-1bff-499e-99c5-077341845ccf response_code = 204 uri = /eip-gateways
19552024-12-13T22:30:45.350ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 174 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:42582 req_id = 50e0cd5c-1004-4426-8e4e-7f04465026d4 response_code = 204 uri = /vpc-routes
19562024-12-13T22:30:45.527ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62717
19572024-12-13T22:30:45.528ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19582024-12-13T22:30:45.566ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19592024-12-13T22:30:45.622ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94608 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:62717 req_id = a1382166-5c85-46f7-a735-f01d37acbfce response_code = 204 uri = /switch-ports
19602024-12-13T22:30:45.622ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62717 req_id = 6ff9a27e-4a2f-4308-a73b-13208b3513d2 response_code = 200 uri = /network-bootstore-config
19612024-12-13T22:30:46.312ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43206
19622024-12-13T22:30:46.312ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 75 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43206 req_id = 62dd77dd-bc7b-4fe0-a795-4626730540e1 response_code = 200 uri = /vpc-routes
19632024-12-13T22:30:46.342ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61882
19642024-12-13T22:30:46.342ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 43 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61882 req_id = 10570241-ce50-4e1d-b481-dcc5c2945ec3 response_code = 200 uri = /vpc-routes
19652024-12-13T22:30:46.381ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
19662024-12-13T22:30:46.381ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43206 req_id = c02acae2-b23a-4c67-8bbc-4b582f5e3749 response_code = 204 uri = /eip-gateways
19672024-12-13T22:30:46.388ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
19682024-12-13T22:30:46.388ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61882 req_id = 8b85c225-701f-4b5f-aeed-9518255ba6ed response_code = 204 uri = /eip-gateways
19692024-12-13T22:30:46.398ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 206 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61882 req_id = 8ffe8e13-9582-481e-9a77-4d0e7c9a07dc response_code = 204 uri = /vpc-routes
19702024-12-13T22:30:46.492ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54567
19712024-12-13T22:30:46.492ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 116 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54567 req_id = 9ea45220-317e-4237-8053-b3296e5e8649 response_code = 200 uri = /v2p
19722024-12-13T22:30:46.498ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 114 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:43206 req_id = 68ca94d3-1917-49ad-92dc-5584a93f19be response_code = 204 uri = /vpc-routes
19732024-12-13T22:30:46.541ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49408
19742024-12-13T22:30:46.541ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49408 req_id = 778a6126-aa22-4e3a-9a1c-7ebffc762248 response_code = 200 uri = /v2p
19752024-12-13T22:30:46.593ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52965
19762024-12-13T22:30:46.594ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19772024-12-13T22:30:46.634ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19782024-12-13T22:30:46.697ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103062 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:52965 req_id = d080fdde-4299-449e-a950-76f985331296 response_code = 204 uri = /switch-ports
19792024-12-13T22:30:46.697ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53998
19802024-12-13T22:30:46.697ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19812024-12-13T22:30:46.701ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3995 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52965 req_id = 366da843-bbae-437b-8429-5b30e90917f6 response_code = 200 uri = /network-bootstore-config
19822024-12-13T22:30:46.735ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19832024-12-13T22:30:46.796ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98900 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:53998 req_id = 906fb696-2389-495f-852c-1562a6bf7b54 response_code = 204 uri = /switch-ports
19842024-12-13T22:30:46.796ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 162 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53998 req_id = 633798f0-a819-4dac-9736-c77968a029c7 response_code = 200 uri = /network-bootstore-config
19852024-12-13T22:31:08.193ZINFOSledAgent (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:62120 remote_addr = [fd00:1122:3344:101::d]:38161 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
19862024-12-13T22:31:08.234ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40064 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = e94ec149-ffa4-4d75-af7e-ffea844748dd response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
19872024-12-13T22:31:14.906ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59436
19882024-12-13T22:31:14.906ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59436 req_id = fdfe61bb-b0ec-44d0-9915-10bc7e4edebe response_code = 200 uri = /vpc-routes
19892024-12-13T22:31:14.920ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
19902024-12-13T22:31:14.920ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:59436 req_id = 5314a9cd-0dc1-4f67-9baf-49cf3050ae5a response_code = 204 uri = /eip-gateways
19912024-12-13T22:31:14.931ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 189 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:59436 req_id = 990ba6e9-97bc-42dc-8c9f-edc779587534 response_code = 204 uri = /vpc-routes
19922024-12-13T22:31:14.981ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37673
19932024-12-13T22:31:14.981ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
19942024-12-13T22:31:15.025ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
19952024-12-13T22:31:15.090ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 109198 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:37673 req_id = 1c096c25-26a9-487b-93a8-13d246568111 response_code = 204 uri = /switch-ports
19962024-12-13T22:31:15.090ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50550
19972024-12-13T22:31:15.091ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 232 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50550 req_id = a69348a0-e22d-4845-b9fd-52ed5e7e8908 response_code = 200 uri = /v2p
19982024-12-13T22:31:15.091ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 233 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37673 req_id = 494b7b72-c373-49c3-80b7-0e40531fa1df response_code = 200 uri = /network-bootstore-config
19992024-12-13T22:31:15.135ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:35413
20002024-12-13T22:31:15.136ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:35413 req_id = af4c6241-e906-49c3-86e6-1c082a4ea63a response_code = 200 uri = /baseboard
20012024-12-13T22:31:16.129ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61715
20022024-12-13T22:31:16.129ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 68 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61715 req_id = aecca7aa-e51e-4737-a81d-1807dbfd40f1 response_code = 200 uri = /vpc-routes
20032024-12-13T22:31:16.184ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
20042024-12-13T22:31:16.184ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 81 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61715 req_id = 02a09810-4345-4a92-b679-ee16859440e9 response_code = 204 uri = /eip-gateways
20052024-12-13T22:31:16.217ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63982
20062024-12-13T22:31:16.217ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 140 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63982 req_id = 49b15270-d4bd-47cc-93dc-4873faab6b02 response_code = 200 uri = /v2p
20072024-12-13T22:31:16.228ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 155 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:61715 req_id = a1ec906a-cbde-49fa-aea7-8472c882096b response_code = 204 uri = /vpc-routes
20082024-12-13T22:31:16.246ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60580
20092024-12-13T22:31:16.246ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 39 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60580 req_id = 8e7f9799-65ec-4f9a-8d46-70e21a7ef07d response_code = 200 uri = /vpc-routes
20102024-12-13T22:31:16.280ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49113
20112024-12-13T22:31:16.281ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20122024-12-13T22:31:16.325ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20132024-12-13T22:31:16.394ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 113652 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:49113 req_id = b98dcbc3-c2fc-4b6b-8373-2393cf116bdf response_code = 204 uri = /switch-ports
20142024-12-13T22:31:16.394ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37667
20152024-12-13T22:31:16.394ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
20162024-12-13T22:31:16.394ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60580 req_id = f45c1f7f-d209-4bef-86e1-8be5081183bb response_code = 204 uri = /eip-gateways
20172024-12-13T22:31:16.395ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20182024-12-13T22:31:16.395ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 253 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49113 req_id = c52dc497-b832-44e5-bd89-89e2461a3444 response_code = 200 uri = /network-bootstore-config
20192024-12-13T22:31:16.405ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 287 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60580 req_id = d8c81b9f-e3b8-4dc9-97d8-0953f6878eaf response_code = 204 uri = /vpc-routes
20202024-12-13T22:31:16.436ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20212024-12-13T22:31:16.445ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45380
20222024-12-13T22:31:16.445ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 158 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45380 req_id = eff9ee77-1f12-464e-929d-468e7b385d3d response_code = 200 uri = /v2p
20232024-12-13T22:31:16.502ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 107162 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:37667 req_id = 18f8d990-f718-46d8-8292-f3007206e75d response_code = 204 uri = /switch-ports
20242024-12-13T22:31:16.502ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 175 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37667 req_id = 97174188-e654-4209-9719-2f94135a73d9 response_code = 200 uri = /network-bootstore-config
20252024-12-13T22:31:38.199ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7455 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = 76966bda-0d62-4805-b141-2bd2c48c990f response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
20262024-12-13T22:31:45.149ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58178
20272024-12-13T22:31:45.149ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58178 req_id = b5d11abe-67f6-4b22-8371-aaee813396c5 response_code = 200 uri = /vpc-routes
20282024-12-13T22:31:45.171ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:58027
20292024-12-13T22:31:45.172ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 30 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:58027 req_id = e61cb329-384b-4db2-9185-bc660cbc37d1 response_code = 200 uri = /baseboard
20302024-12-13T22:31:45.205ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60338
20312024-12-13T22:31:45.205ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60338 req_id = daf00ba7-4c17-492d-a199-b1ff8103c208 response_code = 200 uri = /v2p
20322024-12-13T22:31:45.220ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
20332024-12-13T22:31:45.220ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58178 req_id = 544e7abf-e622-4a4f-b69b-7b0f003ebd7c response_code = 204 uri = /eip-gateways
20342024-12-13T22:31:45.229ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 222 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:58178 req_id = 39e7bfec-ad08-4db5-94ac-db17e2ab1df8 response_code = 204 uri = /vpc-routes
20352024-12-13T22:31:45.320ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36471
20362024-12-13T22:31:45.320ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20372024-12-13T22:31:45.361ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20382024-12-13T22:31:45.416ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95918 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:36471 req_id = b2349a41-e133-42c3-87b9-98b7d6f66225 response_code = 204 uri = /switch-ports
20392024-12-13T22:31:45.416ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36471 req_id = 6642ae41-4cd3-46a3-9c90-90fa988a2b93 response_code = 200 uri = /network-bootstore-config
20402024-12-13T22:31:46.197ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48011
20412024-12-13T22:31:46.197ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 50 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48011 req_id = ab451d9e-2665-4fad-b3c9-e7fcf7522650 response_code = 200 uri = /vpc-routes
20422024-12-13T22:31:46.265ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
20432024-12-13T22:31:46.266ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48011 req_id = 42d39003-3594-453e-ad50-a409a1aedbab response_code = 204 uri = /eip-gateways
20442024-12-13T22:31:46.277ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 257 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48011 req_id = 3c44696c-7247-4869-9ed1-ad02ed4f98d3 response_code = 204 uri = /vpc-routes
20452024-12-13T22:31:46.421ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50146
20462024-12-13T22:31:46.421ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50146 req_id = b9189240-68ca-48fe-a117-bff8158050f8 response_code = 200 uri = /vpc-routes
20472024-12-13T22:31:46.471ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
20482024-12-13T22:31:46.471ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 234 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:50146 req_id = 00649e49-d52e-4335-9c3a-9d5606b95816 response_code = 204 uri = /eip-gateways
20492024-12-13T22:31:46.481ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 204 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:50146 req_id = 917f064e-f17d-412d-9320-673c9adb984c response_code = 204 uri = /vpc-routes
20502024-12-13T22:31:46.487ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55684
20512024-12-13T22:31:46.487ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55684 req_id = 21c0c49c-1818-4934-bfbd-73e3a9aaaf78 response_code = 200 uri = /v2p
20522024-12-13T22:31:46.503ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52805
20532024-12-13T22:31:46.503ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20542024-12-13T22:31:46.546ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20552024-12-13T22:31:46.602ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99279 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:52805 req_id = af3e5928-5a1f-41a7-a0d6-a68e1864e389 response_code = 204 uri = /switch-ports
20562024-12-13T22:31:46.602ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38814
20572024-12-13T22:31:46.603ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 215 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52805 req_id = 09a16a0b-2e15-4524-896b-3a6a7a7370a5 response_code = 200 uri = /network-bootstore-config
20582024-12-13T22:31:46.603ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 197 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38814 req_id = b11ef307-3a9a-46c6-822d-32b3c270cd29 response_code = 200 uri = /v2p
20592024-12-13T22:31:46.641ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56778
20602024-12-13T22:31:46.641ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20612024-12-13T22:31:46.678ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20622024-12-13T22:31:46.736ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95031 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:56778 req_id = 72ee6d0c-a6fe-46b4-9216-06327755d843 response_code = 204 uri = /switch-ports
20632024-12-13T22:31:46.737ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56778 req_id = 8c13ac92-b03c-403b-9af6-68f53511502b response_code = 200 uri = /network-bootstore-config
20642024-12-13T22:32:08.198ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6159 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = cc7c308b-7e24-422a-9c0d-808470e19bd1 response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
20652024-12-13T22:32:14.997ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:46673
20662024-12-13T22:32:14.997ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 80 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:46673 req_id = 4b7c0738-63a5-4f8b-be3d-d06a048b81ed response_code = 200 uri = /vpc-routes
20672024-12-13T22:32:15.053ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
20682024-12-13T22:32:15.053ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46673 req_id = 84d66a34-6484-4b9f-89ea-51e95897292e response_code = 204 uri = /eip-gateways
20692024-12-13T22:32:15.089ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62387
20702024-12-13T22:32:15.089ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 166 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62387 req_id = fad7981c-637d-45be-8bd4-4e84f0a959bd response_code = 200 uri = /v2p
20712024-12-13T22:32:15.099ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 217 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:46673 req_id = a0160b4f-9061-487c-88d9-3273939402d2 response_code = 204 uri = /vpc-routes
20722024-12-13T22:32:15.149ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41188
20732024-12-13T22:32:15.149ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20742024-12-13T22:32:15.191ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20752024-12-13T22:32:15.249ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100156 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:41188 req_id = a36cd87c-9fe6-401d-88f4-7feee27b6a91 response_code = 204 uri = /switch-ports
20762024-12-13T22:32:15.249ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:44609
20772024-12-13T22:32:15.250ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41188 req_id = ef7a7466-b6f5-4b24-aed0-d0350a6e8593 response_code = 200 uri = /network-bootstore-config
20782024-12-13T22:32:15.250ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 63 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:44609 req_id = 49e40bb4-a78b-4fa8-8395-d3a19572bff5 response_code = 200 uri = /baseboard
20792024-12-13T22:32:16.136ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39743
20802024-12-13T22:32:16.136ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39743 req_id = 747900f5-a03c-4940-bdd9-cf1f311e1f00 response_code = 200 uri = /vpc-routes
20812024-12-13T22:32:16.192ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
20822024-12-13T22:32:16.192ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:39743 req_id = e54b2734-f12b-42b6-8e7e-9e34b31333de response_code = 204 uri = /eip-gateways
20832024-12-13T22:32:16.204ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:39743 req_id = 826bc0cd-5c5e-48ef-88a0-6ce09a0545cd response_code = 204 uri = /vpc-routes
20842024-12-13T22:32:16.210ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54602
20852024-12-13T22:32:16.211ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20862024-12-13T22:32:16.252ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20872024-12-13T22:32:16.323ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 112068 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:54602 req_id = b8008663-c292-4218-9325-c84c02a14e23 response_code = 204 uri = /switch-ports
20882024-12-13T22:32:16.323ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61423
20892024-12-13T22:32:16.323ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:33599
20902024-12-13T22:32:16.323ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 259 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54602 req_id = 7479eeba-83eb-4ef8-ac42-989659173a35 response_code = 200 uri = /network-bootstore-config
20912024-12-13T22:32:16.323ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 218 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:33599 req_id = 55e75160-524c-42bb-9c5d-dd83f93d0671 response_code = 200 uri = /v2p
20922024-12-13T22:32:16.323ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61423 req_id = c2a167e4-1a8b-48c9-9523-b182e884d9b6 response_code = 200 uri = /vpc-routes
20932024-12-13T22:32:16.339ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37764
20942024-12-13T22:32:16.339ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
20952024-12-13T22:32:16.379ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
20962024-12-13T22:32:16.441ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101991 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:37764 req_id = dfbb48ae-26d7-4fe8-a70d-a281879c68f4 response_code = 204 uri = /switch-ports
20972024-12-13T22:32:16.441ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63289
20982024-12-13T22:32:16.441ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
20992024-12-13T22:32:16.441ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 154 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61423 req_id = 8289b244-96cf-44ff-82b3-ff219c177931 response_code = 204 uri = /eip-gateways
21002024-12-13T22:32:16.441ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 218 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37764 req_id = 8d639a6f-8601-46a1-8a07-9b1893e7176a response_code = 200 uri = /network-bootstore-config
21012024-12-13T22:32:16.441ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 251 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:63289 req_id = 91c03a0d-040b-4ee8-9c7c-c03e7b54c276 response_code = 200 uri = /v2p
21022024-12-13T22:32:16.451ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 240 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61423 req_id = 9d171a19-668a-4996-8501-69a165dcf3bd response_code = 204 uri = /vpc-routes
21032024-12-13T22:32:27.258ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
21042024-12-13T22:32:38.197ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5939 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = 061eb78d-de26-40a3-a5b5-8b577a43831c response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
21052024-12-13T22:32:44.946ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65320
21062024-12-13T22:32:44.946ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65320 req_id = 8a83af4e-ea06-4ec9-927f-44a6c86640c5 response_code = 200 uri = /vpc-routes
21072024-12-13T22:32:45.009ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
21082024-12-13T22:32:45.009ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:65320 req_id = 3a8da93f-9fe0-4f24-88f6-d7df01b2a954 response_code = 204 uri = /eip-gateways
21092024-12-13T22:32:45.019ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 254 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:65320 req_id = a4d09bd6-2a76-43fd-9f95-1b1a5dfe5493 response_code = 204 uri = /vpc-routes
21102024-12-13T22:32:45.227ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33110
21112024-12-13T22:32:45.227ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 129 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33110 req_id = 19ad8dfe-231a-44e4-9963-fb2239f24c5d response_code = 200 uri = /v2p
21122024-12-13T22:32:45.284ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:59377
21132024-12-13T22:32:45.284ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 43 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:59377 req_id = dcf969fa-2f15-43ef-94de-8e94102bddad response_code = 200 uri = /baseboard
21142024-12-13T22:32:45.429ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:34748
21152024-12-13T22:32:45.429ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21162024-12-13T22:32:45.470ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21172024-12-13T22:32:45.527ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98604 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:34748 req_id = 6bcbb591-0f54-45c3-9d37-cd9deb6cb451 response_code = 204 uri = /switch-ports
21182024-12-13T22:32:45.528ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 115 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:34748 req_id = e9f27493-7319-431a-aa6b-93c5591aebac response_code = 200 uri = /network-bootstore-config
21192024-12-13T22:32:46.283ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54905
21202024-12-13T22:32:46.284ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54905 req_id = b015b7a9-2809-49de-a30b-f7d87d5ee145 response_code = 200 uri = /vpc-routes
21212024-12-13T22:32:46.299ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
21222024-12-13T22:32:46.299ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:54905 req_id = 35ff31d7-0a66-4a13-9cbc-702beef7439a response_code = 204 uri = /eip-gateways
21232024-12-13T22:32:46.407ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37790
21242024-12-13T22:32:46.407ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 124 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37790 req_id = e48221a4-91fb-4eac-b5e4-7b34822414c1 response_code = 200 uri = /v2p
21252024-12-13T22:32:46.411ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 198 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:54905 req_id = 04872a6a-223a-444c-9c98-6458fc2d89b2 response_code = 204 uri = /vpc-routes
21262024-12-13T22:32:46.454ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40845
21272024-12-13T22:32:46.454ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21282024-12-13T22:32:46.496ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21292024-12-13T22:32:46.572ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 118204 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:40845 req_id = a41742e2-29e8-494e-bbe7-39c053febe0b response_code = 204 uri = /switch-ports
21302024-12-13T22:32:46.573ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60153
21312024-12-13T22:32:46.573ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 147 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60153 req_id = fcc485af-8d2b-4218-b0bd-d4b82f7678d1 response_code = 200 uri = /vpc-routes
21322024-12-13T22:32:46.573ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 257 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40845 req_id = 9f462c6c-4d2b-4855-803e-53d4f484751a response_code = 200 uri = /network-bootstore-config
21332024-12-13T22:32:46.627ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
21342024-12-13T22:32:46.627ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60153 req_id = 6a6fb05e-516f-44a2-b7c5-cfa847521daa response_code = 204 uri = /eip-gateways
21352024-12-13T22:32:46.635ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 218 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60153 req_id = caf65996-5dbf-4bbc-8dc3-e7ef0576d795 response_code = 204 uri = /vpc-routes
21362024-12-13T22:32:46.692ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34764
21372024-12-13T22:32:46.692ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21382024-12-13T22:32:46.730ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21392024-12-13T22:32:46.785ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92920 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:34764 req_id = 92dfa3b0-5d65-4f7e-b97f-ac981b14f5f2 response_code = 204 uri = /switch-ports
21402024-12-13T22:32:46.786ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33274
21412024-12-13T22:32:46.786ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 212 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34764 req_id = 7cd12cad-90e4-4eaf-bb89-3bd26519485f response_code = 200 uri = /network-bootstore-config
21422024-12-13T22:32:46.786ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 237 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33274 req_id = 0b8e4535-e6de-4674-a94c-c7a04a33e9d1 response_code = 200 uri = /v2p
21432024-12-13T22:33:08.197ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6840 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = 6d1ca7f1-4689-4552-83ef-201b07a0cf5f response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
21442024-12-13T22:33:14.935ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48426
21452024-12-13T22:33:14.935ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48426 req_id = c13f187f-7951-48a8-b3fa-9e3c9b309809 response_code = 200 uri = /vpc-routes
21462024-12-13T22:33:14.952ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
21472024-12-13T22:33:14.952ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 68 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48426 req_id = 72bb43ef-fef0-43de-803e-7578cd05870c response_code = 204 uri = /eip-gateways
21482024-12-13T22:33:14.963ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65447
21492024-12-13T22:33:14.963ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65447 req_id = 0fa4eb16-d446-437d-bf4d-e485165e2d01 response_code = 200 uri = /v2p
21502024-12-13T22:33:14.977ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 262 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48426 req_id = bd584ab4-75e2-42c9-9ee3-da3ad2011576 response_code = 204 uri = /vpc-routes
21512024-12-13T22:33:15.064ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63612
21522024-12-13T22:33:15.064ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21532024-12-13T22:33:15.105ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21542024-12-13T22:33:15.164ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99712 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:63612 req_id = e5df5916-384c-48f3-9fc8-8d13cb6f0c02 response_code = 204 uri = /switch-ports
21552024-12-13T22:33:15.164ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63612 req_id = 15517281-4f2e-46e5-8c33-1f163897a4f0 response_code = 200 uri = /network-bootstore-config
21562024-12-13T22:33:15.316ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:58174
21572024-12-13T22:33:15.316ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 56 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:58174 req_id = e881ae97-9545-4e02-b1fc-675841df35a5 response_code = 200 uri = /baseboard
21582024-12-13T22:33:16.114ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:34613
21592024-12-13T22:33:16.114ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 55 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:34613 req_id = f6e98ed4-8607-47d4-b398-7274afa87947 response_code = 200 uri = /vpc-routes
21602024-12-13T22:33:16.132ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
21612024-12-13T22:33:16.132ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 68 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:34613 req_id = afb7cea4-24cd-4004-92d0-0a4cf2f2f85b response_code = 204 uri = /eip-gateways
21622024-12-13T22:33:16.140ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 162 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:34613 req_id = 76ab04e9-c191-4fbe-8356-9ec79b899524 response_code = 204 uri = /vpc-routes
21632024-12-13T22:33:16.173ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51824
21642024-12-13T22:33:16.173ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51824 req_id = 5d3d94a3-48c6-4cd8-8937-9af105777680 response_code = 200 uri = /vpc-routes
21652024-12-13T22:33:16.210ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51675
21662024-12-13T22:33:16.210ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 144 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51675 req_id = af15f915-ebb7-4f78-b94f-9b2fd3be740d response_code = 200 uri = /v2p
21672024-12-13T22:33:16.240ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:61826
21682024-12-13T22:33:16.240ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
21692024-12-13T22:33:16.240ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 163 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:51824 req_id = 84437509-6276-46ee-bb9e-85490b9d353f response_code = 204 uri = /eip-gateways
21702024-12-13T22:33:16.240ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21712024-12-13T22:33:16.249ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 382 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:51824 req_id = 0bd62b4e-4cce-4c86-9c5e-063c723e6d39 response_code = 204 uri = /vpc-routes
21722024-12-13T22:33:16.282ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21732024-12-13T22:33:16.333ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:62615
21742024-12-13T22:33:16.343ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102783 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:61826 req_id = 0406ab15-0e19-4ebe-b35f-72d663c31411 response_code = 204 uri = /switch-ports
21752024-12-13T22:33:16.343ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21762024-12-13T22:33:16.348ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 690 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:61826 req_id = 9891f572-6e4d-4a7b-bf57-2044735f48ad response_code = 200 uri = /network-bootstore-config
21772024-12-13T22:33:16.379ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21782024-12-13T22:33:16.406ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44621
21792024-12-13T22:33:16.406ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 155 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44621 req_id = 67271d44-4d36-468b-9080-13889116be8c response_code = 200 uri = /v2p
21802024-12-13T22:33:16.435ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100984 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:62615 req_id = dfedabfa-53e9-4a19-823f-1b5d9baf03d1 response_code = 204 uri = /switch-ports
21812024-12-13T22:33:16.435ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:62615 req_id = 7c69bcfb-36f1-46a5-a1ca-6dada2748810 response_code = 200 uri = /network-bootstore-config
21822024-12-13T22:33:38.197ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6323 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = c720e624-83d9-4148-9678-8774edae3948 response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
21832024-12-13T22:33:44.973ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49093
21842024-12-13T22:33:44.973ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 62 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49093 req_id = 4cb65662-283d-47a9-9eda-9b932cf7793b response_code = 200 uri = /vpc-routes
21852024-12-13T22:33:44.989ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
21862024-12-13T22:33:44.989ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49093 req_id = 11ca4d1a-793a-4222-be1a-8f68b169d0bf response_code = 204 uri = /eip-gateways
21872024-12-13T22:33:45.113ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:58642
21882024-12-13T22:33:45.114ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 142 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:58642 req_id = 5ec3b022-dea9-44fe-8289-b37bdd694f00 response_code = 200 uri = /v2p
21892024-12-13T22:33:45.121ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 195 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49093 req_id = d6bc211c-2f4f-4125-a372-9f44494f0eff response_code = 204 uri = /vpc-routes
21902024-12-13T22:33:45.178ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50200
21912024-12-13T22:33:45.178ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
21922024-12-13T22:33:45.215ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
21932024-12-13T22:33:45.269ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90466 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:50200 req_id = 80610384-54e0-4db6-a842-96a064b14d8e response_code = 204 uri = /switch-ports
21942024-12-13T22:33:45.269ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50200 req_id = 3cbdea41-902f-40cd-942e-2189ab1beeda response_code = 200 uri = /network-bootstore-config
21952024-12-13T22:33:45.349ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:35011
21962024-12-13T22:33:45.349ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:35011 req_id = 200b1039-c075-4198-8873-3527ca4d6259 response_code = 200 uri = /baseboard
21972024-12-13T22:33:46.241ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48233
21982024-12-13T22:33:46.241ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48233 req_id = de8698cd-b225-4568-a3ee-64ef0e4431a2 response_code = 200 uri = /vpc-routes
21992024-12-13T22:33:46.304ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
22002024-12-13T22:33:46.305ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 67 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48233 req_id = 150bf59a-d46d-4911-b26b-4dfbef490049 response_code = 204 uri = /eip-gateways
22012024-12-13T22:33:46.314ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 270 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48233 req_id = bfd143b8-abb9-4856-9341-adacfd9b49a7 response_code = 204 uri = /vpc-routes
22022024-12-13T22:33:46.350ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54949
22032024-12-13T22:33:46.350ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54949 req_id = caa835c3-1fe0-4d79-9095-28e1abec8e97 response_code = 200 uri = /vpc-routes
22042024-12-13T22:33:46.406ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
22052024-12-13T22:33:46.406ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:54949 req_id = 9da92f33-eb17-4edd-bb8a-ab6abf819227 response_code = 204 uri = /eip-gateways
22062024-12-13T22:33:46.463ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51332
22072024-12-13T22:33:46.464ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 122 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51332 req_id = b03190a0-7fce-443f-b926-0335600c279c response_code = 200 uri = /v2p
22082024-12-13T22:33:46.466ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 219 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:54949 req_id = 0dac533d-2281-45ea-8bfc-42a9c02e245d response_code = 204 uri = /vpc-routes
22092024-12-13T22:33:46.476ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47719
22102024-12-13T22:33:46.476ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47719 req_id = 76110e94-6071-4087-a3b1-85baf94644ae response_code = 200 uri = /v2p
22112024-12-13T22:33:46.508ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:36670
22122024-12-13T22:33:46.508ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
22132024-12-13T22:33:46.548ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
22142024-12-13T22:33:46.602ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93200 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:36670 req_id = e13ec720-a444-4c1c-b6bc-9b471254560d response_code = 204 uri = /switch-ports
22152024-12-13T22:33:46.602ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:49374
22162024-12-13T22:33:46.602ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 206 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:36670 req_id = 6912e0d4-e9bf-4a7b-839d-f321cdbaa531 response_code = 200 uri = /network-bootstore-config
22172024-12-13T22:33:46.602ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
22182024-12-13T22:33:46.638ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
22192024-12-13T22:33:46.690ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88242 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:49374 req_id = 6d3a6bfe-4c34-4bca-b5f7-6e26a9dc519d response_code = 204 uri = /switch-ports
22202024-12-13T22:33:46.691ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 149 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:49374 req_id = c160c0ab-f539-47eb-ad46-7c457974329b response_code = 200 uri = /network-bootstore-config
22212024-12-13T22:34:08.197ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6594 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = 5f40143a-d5b4-42c7-86f7-d7432da5017e response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
22222024-12-13T22:34:14.883ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57024
22232024-12-13T22:34:14.883ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57024 req_id = 819e1452-c35d-4c9e-acba-9780e86d34a0 response_code = 200 uri = /vpc-routes
22242024-12-13T22:34:14.926ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
22252024-12-13T22:34:14.926ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57024 req_id = 59e77cfe-238b-4d4e-8c40-aa03c824424c response_code = 204 uri = /eip-gateways
22262024-12-13T22:34:14.934ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 231 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57024 req_id = a90ed016-b5bf-4563-b8ae-5328523a9805 response_code = 204 uri = /vpc-routes
22272024-12-13T22:34:15.004ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64562
22282024-12-13T22:34:15.005ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 120 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64562 req_id = 2fcd8d68-8e9f-4759-afb5-7f0fa20efe97 response_code = 200 uri = /v2p
22292024-12-13T22:34:15.022ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39623
22302024-12-13T22:34:15.022ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
22312024-12-13T22:34:15.063ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
22322024-12-13T22:34:15.118ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95763 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:39623 req_id = 27fe3038-9ba1-4c3f-9c5e-6e61e6ef761a response_code = 204 uri = /switch-ports
22332024-12-13T22:34:15.118ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 122 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39623 req_id = d6c3cc7b-d410-43b5-86be-3edd09172d06 response_code = 200 uri = /network-bootstore-config
22342024-12-13T22:34:15.382ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47443
22352024-12-13T22:34:15.382ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 75 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47443 req_id = dd823f01-c02a-4fa9-9e3e-0ebaba9a6d0d response_code = 200 uri = /baseboard
22362024-12-13T22:34:16.161ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:49161
22372024-12-13T22:34:16.161ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:49161 req_id = 00d05a2d-28c9-4a44-a376-8a108626c369 response_code = 200 uri = /vpc-routes
22382024-12-13T22:34:16.205ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
22392024-12-13T22:34:16.205ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 86 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:49161 req_id = 97182681-7c36-4e51-8e5a-6a7519dff549 response_code = 204 uri = /eip-gateways
22402024-12-13T22:34:16.214ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 213 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:49161 req_id = 959baa66-428a-4281-996e-f7235784ffd0 response_code = 204 uri = /vpc-routes
22412024-12-13T22:34:16.224ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60748
22422024-12-13T22:34:16.224ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60748 req_id = 522a5e39-fe19-4dc2-8851-80f256cffee1 response_code = 200 uri = /vpc-routes
22432024-12-13T22:34:16.266ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
22442024-12-13T22:34:16.266ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60748 req_id = ea928c5f-c9a0-4d81-8f3d-6cfb22eb4069 response_code = 204 uri = /eip-gateways
22452024-12-13T22:34:16.270ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:52931
22462024-12-13T22:34:16.270ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
22472024-12-13T22:34:16.313ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
22482024-12-13T22:34:16.373ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103092 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:52931 req_id = 6a79fb4e-d287-472d-b2ef-fad230075db0 response_code = 204 uri = /switch-ports
22492024-12-13T22:34:16.374ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46792
22502024-12-13T22:34:16.374ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46681
22512024-12-13T22:34:16.374ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 299 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:52931 req_id = 7aefab60-608d-4bea-965e-2a4a16fac38b response_code = 200 uri = /network-bootstore-config
22522024-12-13T22:34:16.374ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 430 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:60748 req_id = 96efef00-ef3d-4377-8237-f8528a7b49ed response_code = 204 uri = /vpc-routes
22532024-12-13T22:34:16.374ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 249 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46681 req_id = 494f8501-e83e-4aac-aff9-27c58225cd5a response_code = 200 uri = /v2p
22542024-12-13T22:34:16.374ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 205 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46792 req_id = c4233591-59e0-47fe-a651-d403a72aa802 response_code = 200 uri = /v2p
22552024-12-13T22:34:16.412ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36691
22562024-12-13T22:34:16.412ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
22572024-12-13T22:34:16.451ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
22582024-12-13T22:34:16.506ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94822 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:36691 req_id = 0a108324-e248-4509-bc21-a48e3efed829 response_code = 204 uri = /switch-ports
22592024-12-13T22:34:16.507ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36691 req_id = 0c1f1338-0b5d-47d4-aef8-c93eee763e0e response_code = 200 uri = /network-bootstore-config
22602024-12-13T22:34:38.196ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6245 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = 78091cdd-bdd9-4743-ab7b-dd29b0e516b5 response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
22612024-12-13T22:34:45.007ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40447
22622024-12-13T22:34:45.008ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40447 req_id = 699fcbb2-de9d-4788-b8b0-286e4c93c3d3 response_code = 200 uri = /vpc-routes
22632024-12-13T22:34:45.058ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
22642024-12-13T22:34:45.058ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 89 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40447 req_id = 8241b194-4258-45de-85f9-ecdeb4033244 response_code = 204 uri = /eip-gateways
22652024-12-13T22:34:45.066ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 209 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:40447 req_id = 48a3c655-6973-4dcf-948f-e3721cd9796e response_code = 204 uri = /vpc-routes
22662024-12-13T22:34:45.159ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39548
22672024-12-13T22:34:45.159ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39548 req_id = 443137e8-8e84-4418-adda-d04d5d47726b response_code = 200 uri = /v2p
22682024-12-13T22:34:45.168ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64751
22692024-12-13T22:34:45.168ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
22702024-12-13T22:34:45.207ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
22712024-12-13T22:34:45.265ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96947 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:64751 req_id = d20b5f21-7e64-4e67-8bbd-a7af4a0e2e24 response_code = 204 uri = /switch-ports
22722024-12-13T22:34:45.265ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39706
22732024-12-13T22:34:45.266ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0e0028b4-c96e-4259-b0d9-2dd1e967b57e, kind: Service { id: 3e0d656e-3552-43ff-8e95-5ed82471e0ae }, name: Name("external-dns-3e0d656e-3552-43ff-8e95-5ed82471e0ae"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 156])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9fbc6464-5658-4709-85f5-cba4ad21c9de, kind: Service { id: 0ef774ca-06b2-40a8-a632-7304d594c07a }, name: Name("external-dns-0ef774ca-06b2-40a8-a632-7304d594c07a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 39])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: bb1ce312-163f-477c-9052-27f110965501, kind: Service { id: ef47bd05-ad7e-42a3-8257-dec06acd2b93 }, name: Name("nexus-ef47bd05-ad7e-42a3-8257-dec06acd2b93"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 50])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: be9ea2e0-22c4-425c-9afb-1597ead9d56b, kind: Service { id: b2389da8-cde2-4174-a8e1-a4dabd361b80 }, name: Name("nexus-b2389da8-cde2-4174-a8e1-a4dabd361b80"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 223, 94])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e47aafa9-7219-497b-8e24-b813038ee602, kind: Service { id: 7e3d78a7-b6c3-4636-b217-b0a79485c981 }, name: Name("nexus-7e3d78a7-b6c3-4636-b217-b0a79485c981"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 163])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
22742024-12-13T22:34:45.266ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 295 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64751 req_id = 8a78fd2d-ecf3-4745-961b-12b38adeb6d5 response_code = 200 uri = /network-bootstore-config
22752024-12-13T22:34:45.266ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
22762024-12-13T22:34:45.266ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
22772024-12-13T22:34:45.266ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
22782024-12-13T22:34:45.266ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22792024-12-13T22:34:45.266ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22802024-12-13T22:34:45.266ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
22812024-12-13T22:34:45.266ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 489 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39706 req_id = e2732ad2-4c98-4cd1-b642-a7e006d26bca response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
22822024-12-13T22:34:45.416ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47395
22832024-12-13T22:34:45.416ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 56 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47395 req_id = e8e53c6e-743c-425d-8de7-524ac66f8470 response_code = 200 uri = /baseboard
22842024-12-13T22:34:46.248ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55093
22852024-12-13T22:34:46.248ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 42 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55093 req_id = e6bdde70-8e97-47cc-b726-69c3113a38e1 response_code = 200 uri = /vpc-routes
22862024-12-13T22:34:46.265ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
22872024-12-13T22:34:46.265ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 63 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:55093 req_id = 1b481479-c89e-4629-a997-b9a3f62c5292 response_code = 204 uri = /eip-gateways
22882024-12-13T22:34:46.274ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 252 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:55093 req_id = f396cf51-aca6-403f-9dec-1ef11e1418be response_code = 204 uri = /vpc-routes
22892024-12-13T22:34:46.386ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42818
22902024-12-13T22:34:46.386ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 157 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42818 req_id = 4fc9eea2-62c9-4412-8396-860edf5e5a24 response_code = 200 uri = /v2p
22912024-12-13T22:34:46.437ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34661
22922024-12-13T22:34:46.438ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34661 req_id = 05d5fde8-1d50-4e78-b460-a11a17d009ea response_code = 200 uri = /vpc-routes
22932024-12-13T22:34:46.451ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
22942024-12-13T22:34:46.451ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34661 req_id = 2a25462a-8ef6-4760-bb64-86fc5a7857ca response_code = 204 uri = /eip-gateways
22952024-12-13T22:34:46.467ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47387
22962024-12-13T22:34:46.467ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0e0028b4-c96e-4259-b0d9-2dd1e967b57e, kind: Service { id: 3e0d656e-3552-43ff-8e95-5ed82471e0ae }, name: Name("external-dns-3e0d656e-3552-43ff-8e95-5ed82471e0ae"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 156])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9fbc6464-5658-4709-85f5-cba4ad21c9de, kind: Service { id: 0ef774ca-06b2-40a8-a632-7304d594c07a }, name: Name("external-dns-0ef774ca-06b2-40a8-a632-7304d594c07a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 39])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: bb1ce312-163f-477c-9052-27f110965501, kind: Service { id: ef47bd05-ad7e-42a3-8257-dec06acd2b93 }, name: Name("nexus-ef47bd05-ad7e-42a3-8257-dec06acd2b93"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 50])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: be9ea2e0-22c4-425c-9afb-1597ead9d56b, kind: Service { id: b2389da8-cde2-4174-a8e1-a4dabd361b80 }, name: Name("nexus-b2389da8-cde2-4174-a8e1-a4dabd361b80"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 223, 94])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e47aafa9-7219-497b-8e24-b813038ee602, kind: Service { id: 7e3d78a7-b6c3-4636-b217-b0a79485c981 }, name: Name("nexus-7e3d78a7-b6c3-4636-b217-b0a79485c981"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 163])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
22972024-12-13T22:34:46.467ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
22982024-12-13T22:34:46.467ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
22992024-12-13T22:34:46.467ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23002024-12-13T22:34:46.467ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23012024-12-13T22:34:46.467ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23022024-12-13T22:34:46.467ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23032024-12-13T22:34:46.467ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 355 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:47387 req_id = f7862a4c-67af-47f5-ac51-88c97e977566 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23042024-12-13T22:34:46.487ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34921
23052024-12-13T22:34:46.487ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 255 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34921 req_id = 285a81d5-8549-401a-9efe-ff252278e426 response_code = 200 uri = /v2p
23062024-12-13T22:34:46.496ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 169 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:34661 req_id = de1139c6-364a-451a-9a58-0c17e41001c1 response_code = 204 uri = /vpc-routes
23072024-12-13T22:34:46.510ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40535
23082024-12-13T22:34:46.510ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
23092024-12-13T22:34:46.551ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
23102024-12-13T22:34:46.611ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101148 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:40535 req_id = 0b3b13a8-2d44-402a-b37e-b9e0c6f0adda response_code = 204 uri = /switch-ports
23112024-12-13T22:34:46.611ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40005
23122024-12-13T22:34:46.611ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
23132024-12-13T22:34:46.614ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3369 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40535 req_id = eacf28c4-fd04-4341-a728-9c11b1c9ee70 response_code = 200 uri = /network-bootstore-config
23142024-12-13T22:34:46.617ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56393
23152024-12-13T22:34:46.618ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0e0028b4-c96e-4259-b0d9-2dd1e967b57e, kind: Service { id: 3e0d656e-3552-43ff-8e95-5ed82471e0ae }, name: Name("external-dns-3e0d656e-3552-43ff-8e95-5ed82471e0ae"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 156])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9fbc6464-5658-4709-85f5-cba4ad21c9de, kind: Service { id: 0ef774ca-06b2-40a8-a632-7304d594c07a }, name: Name("external-dns-0ef774ca-06b2-40a8-a632-7304d594c07a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 39])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: bb1ce312-163f-477c-9052-27f110965501, kind: Service { id: ef47bd05-ad7e-42a3-8257-dec06acd2b93 }, name: Name("nexus-ef47bd05-ad7e-42a3-8257-dec06acd2b93"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 50])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: be9ea2e0-22c4-425c-9afb-1597ead9d56b, kind: Service { id: b2389da8-cde2-4174-a8e1-a4dabd361b80 }, name: Name("nexus-b2389da8-cde2-4174-a8e1-a4dabd361b80"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 223, 94])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e47aafa9-7219-497b-8e24-b813038ee602, kind: Service { id: 7e3d78a7-b6c3-4636-b217-b0a79485c981 }, name: Name("nexus-7e3d78a7-b6c3-4636-b217-b0a79485c981"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 163])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
23162024-12-13T22:34:46.618ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23172024-12-13T22:34:46.618ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
23182024-12-13T22:34:46.618ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
23192024-12-13T22:34:46.618ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23202024-12-13T22:34:46.618ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23212024-12-13T22:34:46.618ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
23222024-12-13T22:34:46.618ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 398 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:56393 req_id = a332c300-bcc6-400a-a3d7-ec33d3cf73f8 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23232024-12-13T22:34:46.645ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
23242024-12-13T22:34:46.703ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91885 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:40005 req_id = c150dcfd-429f-4c80-ba95-dcc4a50bed6a response_code = 204 uri = /switch-ports
23252024-12-13T22:34:46.703ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 140 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40005 req_id = 3028918d-e76b-45c4-9e80-f8e6a60b3033 response_code = 200 uri = /network-bootstore-config
23262024-12-13T22:35:08.200ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9806 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = bd4a2701-9cee-4d98-a155-7cda3d9c7e76 response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
23272024-12-13T22:35:14.887ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51873
23282024-12-13T22:35:14.887ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51873 req_id = 9339dca6-54fe-4deb-aad1-e28c65f12bc8 response_code = 200 uri = /vpc-routes
23292024-12-13T22:35:14.934ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
23302024-12-13T22:35:14.934ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51873 req_id = 965d6498-7c0e-44d3-89af-1b0bac00f951 response_code = 204 uri = /eip-gateways
23312024-12-13T22:35:14.942ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 231 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:51873 req_id = 0db0c93b-c666-424e-8aa4-54e34a5b9f1a response_code = 204 uri = /vpc-routes
23322024-12-13T22:35:14.967ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48858
23332024-12-13T22:35:14.967ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
23342024-12-13T22:35:15.008ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
23352024-12-13T22:35:15.073ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105916 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:48858 req_id = 9d3be455-8473-4693-baf9-69aa31c5ff6a response_code = 204 uri = /switch-ports
23362024-12-13T22:35:15.073ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51333
23372024-12-13T22:35:15.073ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 185 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48858 req_id = d6a09097-9f96-4c37-86d7-dc321018c4f8 response_code = 200 uri = /network-bootstore-config
23382024-12-13T22:35:15.074ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 792 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51333 req_id = 9b3bdb4b-92ae-47e9-9331-415d6a933274 response_code = 200 uri = /v2p
23392024-12-13T22:35:15.449ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:48078
23402024-12-13T22:35:15.449ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 82 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:48078 req_id = eff90c3c-b978-4a97-87ae-2f6faad87f3b response_code = 200 uri = /baseboard
23412024-12-13T22:35:16.055ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50009
23422024-12-13T22:35:16.056ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 118 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50009 req_id = ddb36e7c-72f8-4e8f-a001-e27e02c3c504 response_code = 200 uri = /vpc-routes
23432024-12-13T22:35:16.125ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
23442024-12-13T22:35:16.125ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:50009 req_id = b87b9e7e-b4e6-4959-a418-e272871d6fef response_code = 204 uri = /eip-gateways
23452024-12-13T22:35:16.134ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 333 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:50009 req_id = 247ba230-ccce-4997-b2b5-336e40d17864 response_code = 204 uri = /vpc-routes
23462024-12-13T22:35:16.248ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:64307
23472024-12-13T22:35:16.248ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:64307 req_id = 7b9ff505-048c-4d67-9146-250fc2b489d5 response_code = 200 uri = /v2p
23482024-12-13T22:35:16.258ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53064
23492024-12-13T22:35:16.258ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 23 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53064 req_id = c99a2ce1-8d9a-4398-83be-7962130c0afe response_code = 200 uri = /vpc-routes
23502024-12-13T22:35:16.259ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:41917
23512024-12-13T22:35:16.259ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
23522024-12-13T22:35:16.303ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
23532024-12-13T22:35:16.364ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104852 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:41917 req_id = 8e087e66-3e7c-492d-983b-7a076c13a9e0 response_code = 204 uri = /switch-ports
23542024-12-13T22:35:16.364ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43877
23552024-12-13T22:35:16.364ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
23562024-12-13T22:35:16.365ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53064 req_id = 896d4425-cde5-4766-b23e-f39612c4ff65 response_code = 204 uri = /eip-gateways
23572024-12-13T22:35:16.365ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 192 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43877 req_id = 002bcca0-08d4-4ec6-852f-35ed22771cca response_code = 200 uri = /v2p
23582024-12-13T22:35:16.365ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 229 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:41917 req_id = bcda3bd2-00ef-472e-8c79-e82892834bdc response_code = 200 uri = /network-bootstore-config
23592024-12-13T22:35:16.374ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 302 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:53064 req_id = f8130964-9f6e-4497-bf69-2ddcb9b486ed response_code = 204 uri = /vpc-routes
23602024-12-13T22:35:16.396ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40967
23612024-12-13T22:35:16.396ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
23622024-12-13T22:35:16.435ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
23632024-12-13T22:35:16.496ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99431 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:40967 req_id = ac9ac8d4-aa4c-4b47-a923-d27bbc650869 response_code = 204 uri = /switch-ports
23642024-12-13T22:35:16.496ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40967 req_id = 4407c52d-40da-4365-ad9c-25384463ecd1 response_code = 200 uri = /network-bootstore-config
23652024-12-13T22:35:38.195ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 5368 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = b021f5ee-1f37-4705-8ff6-7d3f9eb39242 response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
23662024-12-13T22:35:45.015ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:47739
23672024-12-13T22:35:45.015ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:47739 req_id = f0c9a9c5-8637-40e4-9286-331296345ab8 response_code = 200 uri = /vpc-routes
23682024-12-13T22:35:45.031ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
23692024-12-13T22:35:45.031ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47739 req_id = 29ef9b91-d946-4986-a26c-264799343938 response_code = 204 uri = /eip-gateways
23702024-12-13T22:35:45.040ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 239 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:47739 req_id = 21764b70-624e-44d1-905b-31bd1743c522 response_code = 204 uri = /vpc-routes
23712024-12-13T22:35:45.236ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43563
23722024-12-13T22:35:45.236ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 131 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43563 req_id = e9fbf65f-e6a6-48d7-8f11-0ba923288f9d response_code = 200 uri = /v2p
23732024-12-13T22:35:45.335ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63612
23742024-12-13T22:35:45.335ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
23752024-12-13T22:35:45.377ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
23762024-12-13T22:35:45.433ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98005 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:63612 req_id = 0a65e919-8a24-4041-8849-8b2465695714 response_code = 204 uri = /switch-ports
23772024-12-13T22:35:45.433ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63612 req_id = a0a1ec2d-881d-41e3-a95e-e5a75740b850 response_code = 200 uri = /network-bootstore-config
23782024-12-13T22:35:45.482ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:62465
23792024-12-13T22:35:45.483ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 75 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:62465 req_id = 36c4d5b8-ffab-4bb2-a493-f5a3d525b916 response_code = 200 uri = /baseboard
23802024-12-13T22:35:46.211ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62991
23812024-12-13T22:35:46.211ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62991 req_id = 60dcd380-951c-4d3c-a6e3-94f712e9237d response_code = 200 uri = /vpc-routes
23822024-12-13T22:35:46.261ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
23832024-12-13T22:35:46.261ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:62991 req_id = 4fad60e0-a051-46c0-8c1f-14d2d8459fbb response_code = 204 uri = /eip-gateways
23842024-12-13T22:35:46.270ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 263 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:62991 req_id = eff055fd-8eaa-46bc-b5ce-1efed306a761 response_code = 204 uri = /vpc-routes
23852024-12-13T22:35:46.380ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60746
23862024-12-13T22:35:46.381ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 133 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60746 req_id = 38079194-88d9-45d4-9bb0-8f95947349f0 response_code = 200 uri = /v2p
23872024-12-13T22:35:46.467ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:45243
23882024-12-13T22:35:46.467ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 40 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:45243 req_id = cc6f59bd-2b5d-405c-aaa8-e409a6622484 response_code = 200 uri = /vpc-routes
23892024-12-13T22:35:46.514ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
23902024-12-13T22:35:46.514ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:45243 req_id = 15cf0553-9f0c-4f6f-bb7d-961bb58b6140 response_code = 204 uri = /eip-gateways
23912024-12-13T22:35:46.525ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 258 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:45243 req_id = 931436a1-9b3b-4766-9e16-4126e1fb899a response_code = 204 uri = /vpc-routes
23922024-12-13T22:35:46.546ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45234
23932024-12-13T22:35:46.546ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
23942024-12-13T22:35:46.587ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
23952024-12-13T22:35:46.647ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100893 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:45234 req_id = b2a06df0-bff8-4639-9d31-eca1fb6d363b response_code = 204 uri = /switch-ports
23962024-12-13T22:35:46.647ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45234 req_id = 22df603b-74e8-42b9-8ef6-ac9c3318e4cb response_code = 200 uri = /network-bootstore-config
23972024-12-13T22:35:46.684ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:54370
23982024-12-13T22:35:46.684ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:54370 req_id = eb9654d1-7d88-4b21-8c71-94ee6d8fbf5a response_code = 200 uri = /v2p
23992024-12-13T22:35:46.816ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:40651
24002024-12-13T22:35:46.816ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
24012024-12-13T22:35:46.857ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
24022024-12-13T22:35:46.914ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 97926 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:40651 req_id = 6ce77435-9765-48c2-9e54-bff40b10fb3e response_code = 204 uri = /switch-ports
24032024-12-13T22:35:46.917ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:40651 req_id = 2aa85d0f-75f7-48d7-94a8-337149911aea response_code = 200 uri = /network-bootstore-config
24042024-12-13T22:36:08.199ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8862 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = b80078db-f702-47f0-9b91-0b9571cb87a8 response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
24052024-12-13T22:36:14.975ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:57252
24062024-12-13T22:36:14.975ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:57252 req_id = cf0d87d9-52f6-4589-8e28-7300dcee4590 response_code = 200 uri = /vpc-routes
24072024-12-13T22:36:14.992ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
24082024-12-13T22:36:14.992ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57252 req_id = e357028d-1651-486d-91d3-b9d129a6e536 response_code = 204 uri = /eip-gateways
24092024-12-13T22:36:15.021ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40802
24102024-12-13T22:36:15.022ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 134 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40802 req_id = df8b5c6e-ccd8-48a2-8bff-184f0dcdd106 response_code = 200 uri = /v2p
24112024-12-13T22:36:15.032ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 227 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:57252 req_id = 86f50381-18a4-44c0-8e65-e3758c5c1d04 response_code = 204 uri = /vpc-routes
24122024-12-13T22:36:15.071ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33009
24132024-12-13T22:36:15.071ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
24142024-12-13T22:36:15.109ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
24152024-12-13T22:36:15.172ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100401 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:33009 req_id = cda80484-a572-420c-839b-78e45085cd02 response_code = 204 uri = /switch-ports
24162024-12-13T22:36:15.172ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33009 req_id = 4baf10fb-1f91-431a-8e24-7671dfadcb6a response_code = 200 uri = /network-bootstore-config
24172024-12-13T22:36:15.516ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:47939
24182024-12-13T22:36:15.516ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:47939 req_id = 8ab75819-8320-49a7-885b-bd6088948de3 response_code = 200 uri = /baseboard
24192024-12-13T22:36:16.190ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:60577
24202024-12-13T22:36:16.190ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:60577 req_id = fcdaeb54-a592-43b1-a920-4b3444c664d7 response_code = 200 uri = /vpc-routes
24212024-12-13T22:36:16.211ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
24222024-12-13T22:36:16.211ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:60577 req_id = d1551cde-a375-4273-9d5c-7a609235b06a response_code = 204 uri = /eip-gateways
24232024-12-13T22:36:16.220ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 251 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:60577 req_id = 39700265-c7fc-4b66-872f-7d06f548046b response_code = 204 uri = /vpc-routes
24242024-12-13T22:36:16.281ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38312
24252024-12-13T22:36:16.281ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38312 req_id = a222ed80-414c-454e-aaab-4b7162c732f3 response_code = 200 uri = /v2p
24262024-12-13T22:36:16.289ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54889
24272024-12-13T22:36:16.290ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
24282024-12-13T22:36:16.328ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
24292024-12-13T22:36:16.388ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98212 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:54889 req_id = 15838b6a-5853-424a-aecf-73c3c139549a response_code = 204 uri = /switch-ports
24302024-12-13T22:36:16.388ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56057
24312024-12-13T22:36:16.388ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61877
24322024-12-13T22:36:16.388ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 284 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54889 req_id = 5be6ac30-6353-4ea7-ad5b-018a545661d6 response_code = 200 uri = /network-bootstore-config
24332024-12-13T22:36:16.388ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61877 req_id = b9430c5c-443c-428e-a538-07d38cd8392e response_code = 200 uri = /vpc-routes
24342024-12-13T22:36:16.388ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 225 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56057 req_id = d32b646b-a3e7-4e34-84ba-423e7bb06117 response_code = 200 uri = /v2p
24352024-12-13T22:36:16.403ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
24362024-12-13T22:36:16.403ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 151 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61877 req_id = fb4ba366-5c1d-405d-9fa7-1d4095e01925 response_code = 204 uri = /eip-gateways
24372024-12-13T22:36:16.410ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34194
24382024-12-13T22:36:16.410ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
24392024-12-13T22:36:16.449ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
24402024-12-13T22:36:16.506ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95432 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:34194 req_id = 1c896932-6f5f-4233-aed6-ba02888a7e0e response_code = 204 uri = /switch-ports
24412024-12-13T22:36:16.506ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 298 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34194 req_id = 538ec17e-c642-4367-93d3-7e8ee5bc06db response_code = 200 uri = /network-bootstore-config
24422024-12-13T22:36:16.506ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 355 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:61877 req_id = ba7c8e02-c977-4953-a585-0615f4c86b6f response_code = 204 uri = /vpc-routes
24432024-12-13T22:36:38.197ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8012 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = dbe51ee5-e596-4e4e-a036-de296dc1cea8 response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
24442024-12-13T22:36:44.879ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:33478
24452024-12-13T22:36:44.879ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:33478 req_id = 2e5ca16e-c178-42e8-bd19-ee82b39e9adb response_code = 200 uri = /vpc-routes
24462024-12-13T22:36:44.956ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
24472024-12-13T22:36:44.956ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 111 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:33478 req_id = b558f996-a29d-45fa-ab64-32eae2032ee2 response_code = 204 uri = /eip-gateways
24482024-12-13T22:36:44.966ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 312 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:33478 req_id = a4064b0d-6723-4582-9e90-fb1cb0f04e64 response_code = 204 uri = /vpc-routes
24492024-12-13T22:36:45.231ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:65283
24502024-12-13T22:36:45.231ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 149 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:65283 req_id = 11602004-fa9c-45de-95ae-b76e1571579a response_code = 200 uri = /v2p
24512024-12-13T22:36:45.298ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63906
24522024-12-13T22:36:45.298ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
24532024-12-13T22:36:45.339ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
24542024-12-13T22:36:45.399ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101516 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:63906 req_id = 231122ae-5b73-41cd-8e29-fc5bde6ac380 response_code = 204 uri = /switch-ports
24552024-12-13T22:36:45.400ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63906 req_id = c2a3a163-9d1c-4c82-ac23-7740d653d4e8 response_code = 200 uri = /network-bootstore-config
24562024-12-13T22:36:45.550ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:56960
24572024-12-13T22:36:45.550ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 70 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:56960 req_id = be853d7c-365f-411f-a658-87bea785ad25 response_code = 200 uri = /baseboard
24582024-12-13T22:36:46.278ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53816
24592024-12-13T22:36:46.278ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53816 req_id = 18d34664-2884-49cc-8abb-888965906450 response_code = 200 uri = /vpc-routes
24602024-12-13T22:36:46.301ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:37164
24612024-12-13T22:36:46.301ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 104 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:37164 req_id = 43cb4cd4-df2d-4450-84b7-5c57cd893d5c response_code = 200 uri = /vpc-routes
24622024-12-13T22:36:46.348ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
24632024-12-13T22:36:46.348ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53816 req_id = 837e80f1-81f1-4ff5-8b64-3926f9f7c68b response_code = 204 uri = /eip-gateways
24642024-12-13T22:36:46.368ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 199 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53816 req_id = ab86b23c-c8c5-4758-8b75-73e504751636 response_code = 204 uri = /vpc-routes
24652024-12-13T22:36:46.376ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
24662024-12-13T22:36:46.376ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 39 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37164 req_id = 3b416681-7cb7-4e06-ae7f-6498029c4824 response_code = 204 uri = /eip-gateways
24672024-12-13T22:36:46.389ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 204 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:37164 req_id = 6a54d4bb-44c3-48fc-883f-0c926504530b response_code = 204 uri = /vpc-routes
24682024-12-13T22:36:46.436ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39021
24692024-12-13T22:36:46.436ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
24702024-12-13T22:36:46.479ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
24712024-12-13T22:36:46.534ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98167 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:39021 req_id = a2ece3e1-f531-49e1-a732-a1668915ba45 response_code = 204 uri = /switch-ports
24722024-12-13T22:36:46.534ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:58468
24732024-12-13T22:36:46.535ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 214 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:58468 req_id = 792e1970-59ff-4a34-94cd-23bdb13d7c3d response_code = 200 uri = /v2p
24742024-12-13T22:36:46.535ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 245 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39021 req_id = 1a1e4ae1-a9e1-4c43-b5ca-7851cdc28b7d response_code = 200 uri = /network-bootstore-config
24752024-12-13T22:36:46.545ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:65135
24762024-12-13T22:36:46.546ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:65135 req_id = 51a3ac68-5b2d-4eb3-82e6-38a4c4e5952d response_code = 200 uri = /v2p
24772024-12-13T22:36:46.603ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58120
24782024-12-13T22:36:46.603ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
24792024-12-13T22:36:46.644ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
24802024-12-13T22:36:46.703ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100767 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:58120 req_id = 2d43e4be-35fd-4bc8-93b5-803df0b6c814 response_code = 204 uri = /switch-ports
24812024-12-13T22:36:46.709ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58120 req_id = 7a69be5d-f217-43cb-a1a6-d70e0dbc7388 response_code = 200 uri = /network-bootstore-config
24822024-12-13T22:37:08.196ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7514 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = 7ecf554d-ee5a-49bb-9736-91d47accb01c response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
24832024-12-13T22:37:09.270ZINFOSledAgent (auto-cleanup-task): running automatic periodic zone bundle cleanup file = sled-agent/src/zone_bundle.rs:197
24842024-12-13T22:37:09.270ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
24852024-12-13T22:37:09.304ZINFOSledAgent (auto-cleanup-task): top of bundle cleanup loop file = sled-agent/src/zone_bundle.rs:186 next_cleanup = Instant { tv_sec: 1973, tv_nsec: 486949043 } time_to_next_cleanup = 599.99999441s
24862024-12-13T22:37:14.920ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62290
24872024-12-13T22:37:14.920ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62290 req_id = 080309bc-760b-4aec-b4b2-4f2d603a3868 response_code = 200 uri = /vpc-routes
24882024-12-13T22:37:14.966ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
24892024-12-13T22:37:14.966ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 93 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:62290 req_id = 43a52d78-b955-4a6c-8010-e2469c9e1d86 response_code = 204 uri = /eip-gateways
24902024-12-13T22:37:14.976ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 286 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:62290 req_id = 1438fb21-5457-4cd7-b975-d332a6e390db response_code = 204 uri = /vpc-routes
24912024-12-13T22:37:15.030ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:54135
24922024-12-13T22:37:15.030ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
24932024-12-13T22:37:15.071ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
24942024-12-13T22:37:15.129ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98596 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:54135 req_id = e58cdaa6-c4f7-4705-8282-36cf84b7348f response_code = 204 uri = /switch-ports
24952024-12-13T22:37:15.129ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:43927
24962024-12-13T22:37:15.129ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 194 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:43927 req_id = cfc1ffd3-73dc-4f25-b403-cd5876c1c652 response_code = 200 uri = /v2p
24972024-12-13T22:37:15.129ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 269 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:54135 req_id = 0ae755e0-8a63-49c8-9611-37205e6dac39 response_code = 200 uri = /network-bootstore-config
24982024-12-13T22:37:15.582ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:36893
24992024-12-13T22:37:15.582ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 46 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:36893 req_id = 69cf384c-c850-4fe0-b300-52d9d396b9ee response_code = 200 uri = /baseboard
25002024-12-13T22:37:16.185ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:48696
25012024-12-13T22:37:16.185ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:48696 req_id = ba84d457-49ef-4108-ad93-329d84e1de36 response_code = 200 uri = /vpc-routes
25022024-12-13T22:37:16.233ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57665
25032024-12-13T22:37:16.233ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57665 req_id = 5afe661b-88a4-445f-bafe-a1bca670eed5 response_code = 200 uri = /v2p
25042024-12-13T22:37:16.235ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
25052024-12-13T22:37:16.235ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 61 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48696 req_id = 0447c080-d9c6-4917-be36-fd617bb97f37 response_code = 204 uri = /eip-gateways
25062024-12-13T22:37:16.244ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 297 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:48696 req_id = a327c50b-39f0-412d-a37b-693ae10ac02f response_code = 204 uri = /vpc-routes
25072024-12-13T22:37:16.290ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:44992
25082024-12-13T22:37:16.290ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 35 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:44992 req_id = 620021d9-7033-4e6a-84ca-472b15571d89 response_code = 200 uri = /vpc-routes
25092024-12-13T22:37:16.310ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:50412
25102024-12-13T22:37:16.310ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 80 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:50412 req_id = a9f28fc4-f828-48e0-89aa-49b9a7c7b997 response_code = 200 uri = /v2p
25112024-12-13T22:37:16.312ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39155
25122024-12-13T22:37:16.312ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
25132024-12-13T22:37:16.351ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
25142024-12-13T22:37:16.410ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98674 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:39155 req_id = abb2f2ef-5dba-4c1c-8895-94a5ea11eaaf response_code = 204 uri = /switch-ports
25152024-12-13T22:37:16.410ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:48955
25162024-12-13T22:37:16.411ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
25172024-12-13T22:37:16.411ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
25182024-12-13T22:37:16.411ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 245 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39155 req_id = 2f3246b8-5bb7-4483-8443-b0fb97df7cd2 response_code = 200 uri = /network-bootstore-config
25192024-12-13T22:37:16.414ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3689 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44992 req_id = 84474503-06ad-47ae-84e5-1dccc5d062f1 response_code = 204 uri = /eip-gateways
25202024-12-13T22:37:16.428ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1007 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:44992 req_id = c8181170-df49-4e0d-ab20-380a94ff023f response_code = 204 uri = /vpc-routes
25212024-12-13T22:37:16.450ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
25222024-12-13T22:37:16.503ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92705 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:48955 req_id = e3be0da8-9c92-4198-b061-733b09238fce response_code = 204 uri = /switch-ports
25232024-12-13T22:37:16.504ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 145 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:48955 req_id = e8639dc3-f528-466f-b47f-5919ddb4c17b response_code = 200 uri = /network-bootstore-config
25242024-12-13T22:37:27.256ZINFOSledAgent (DumpSetup-worker): No archival destination for crash dumps yet chosen. file = sled-agent/src/dump_setup.rs:862
25252024-12-13T22:37:38.195ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6972 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = d15716f3-2e4b-46d1-8c16-28ae626d8925 response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
25262024-12-13T22:37:45.118ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:39423
25272024-12-13T22:37:45.118ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 77 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:39423 req_id = b3885e4b-49ca-48e5-a24b-a3aa5393e896 response_code = 200 uri = /vpc-routes
25282024-12-13T22:37:45.135ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
25292024-12-13T22:37:45.135ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39423 req_id = 9c94749d-35ce-4ce4-ad72-6cceec7efb1a response_code = 204 uri = /eip-gateways
25302024-12-13T22:37:45.143ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 195 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:39423 req_id = 1c4e1e46-b437-424a-a559-49dc1252233e response_code = 204 uri = /vpc-routes
25312024-12-13T22:37:45.284ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:38050
25322024-12-13T22:37:45.285ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:38050 req_id = 88a049fe-783f-45fe-8b10-f3ccf18e735d response_code = 200 uri = /v2p
25332024-12-13T22:37:45.351ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:51892
25342024-12-13T22:37:45.351ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
25352024-12-13T22:37:45.390ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
25362024-12-13T22:37:45.446ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95498 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:51892 req_id = 8125bbc4-ffc9-4f4a-94d1-d5ed695d1710 response_code = 204 uri = /switch-ports
25372024-12-13T22:37:45.446ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:51892 req_id = 923017de-85d8-44ed-b710-1a189950f911 response_code = 200 uri = /network-bootstore-config
25382024-12-13T22:37:45.617ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:65109
25392024-12-13T22:37:45.617ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 80 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:65109 req_id = 37b74ab9-d902-40db-b372-0a60a815cd2e response_code = 200 uri = /baseboard
25402024-12-13T22:37:46.302ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:56996
25412024-12-13T22:37:46.302ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 69 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:56996 req_id = d0d6b725-963e-4b2b-8ebb-f74fb8425f29 response_code = 200 uri = /vpc-routes
25422024-12-13T22:37:46.361ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
25432024-12-13T22:37:46.361ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 84 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:56996 req_id = 8f553923-ce19-4212-822d-c74fea773b77 response_code = 204 uri = /eip-gateways
25442024-12-13T22:37:46.368ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:64448
25452024-12-13T22:37:46.368ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 29 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:64448 req_id = 73ff57df-93db-4b27-8e86-bbc6d3e0cb4a response_code = 200 uri = /vpc-routes
25462024-12-13T22:37:46.370ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 173 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:56996 req_id = cd30623c-9a31-4ee1-bf7c-057b95ec4e73 response_code = 204 uri = /vpc-routes
25472024-12-13T22:37:46.383ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
25482024-12-13T22:37:46.383ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 42 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64448 req_id = df780b68-7dc2-4694-b48d-ad13f2b16619 response_code = 204 uri = /eip-gateways
25492024-12-13T22:37:46.392ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 158 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:64448 req_id = 55c44bb9-ea69-4e91-b8b2-881c123d3b65 response_code = 204 uri = /vpc-routes
25502024-12-13T22:37:46.406ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:37892
25512024-12-13T22:37:46.406ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
25522024-12-13T22:37:46.448ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
25532024-12-13T22:37:46.512ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106434 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:37892 req_id = 5f8b5490-d531-423b-bb4a-75c3a9cae31b response_code = 204 uri = /switch-ports
25542024-12-13T22:37:46.513ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:37892 req_id = 02cef280-574f-4890-9b7d-995b3a6921eb response_code = 200 uri = /network-bootstore-config
25552024-12-13T22:37:46.520ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41738
25562024-12-13T22:37:46.520ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41738 req_id = 54aee78e-595d-4e4e-ac8a-e10339c7c01b response_code = 200 uri = /v2p
25572024-12-13T22:37:46.548ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:47390
25582024-12-13T22:37:46.549ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:47390 req_id = 3d4d4f7d-fe7a-4a5a-b3c4-98348262147f response_code = 200 uri = /v2p
25592024-12-13T22:37:46.593ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:41006
25602024-12-13T22:37:46.593ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
25612024-12-13T22:37:46.633ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
25622024-12-13T22:37:46.688ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95088 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:41006 req_id = 23bea5c6-a795-4d08-8781-aaa33665690e response_code = 204 uri = /switch-ports
25632024-12-13T22:37:46.688ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:41006 req_id = 58d31a15-f92a-4fec-b113-62ed36720df0 response_code = 200 uri = /network-bootstore-config
25642024-12-13T22:38:08.194ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 6521 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = 0f84339c-b83a-4c5d-9beb-fd61eff2fc7a response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
25652024-12-13T22:38:14.899ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:49335
25662024-12-13T22:38:14.900ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:49335 req_id = ea7d55aa-b391-4dae-8b18-120bd98f6342 response_code = 200 uri = /vpc-routes
25672024-12-13T22:38:14.951ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:63056
25682024-12-13T22:38:14.951ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:63056 req_id = 68f0e483-a8fc-41c8-86f7-84c46cbdd69e response_code = 200 uri = /v2p
25692024-12-13T22:38:14.993ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
25702024-12-13T22:38:14.993ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49335 req_id = 24e5b44e-aed3-4735-ada7-f4d15a2bc4ac response_code = 204 uri = /eip-gateways
25712024-12-13T22:38:15.003ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 271 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:49335 req_id = 19a02133-6ee9-440e-9f01-24ee070e5266 response_code = 204 uri = /vpc-routes
25722024-12-13T22:38:15.007ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59486
25732024-12-13T22:38:15.007ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
25742024-12-13T22:38:15.052ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
25752024-12-13T22:38:15.115ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108344 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59486 req_id = b7e51c2e-8231-48c5-b672-87647d905c1b response_code = 204 uri = /switch-ports
25762024-12-13T22:38:15.116ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59486 req_id = f3b7ecc5-3512-442c-bd58-73953c490088 response_code = 200 uri = /network-bootstore-config
25772024-12-13T22:38:15.651ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:59226
25782024-12-13T22:38:15.651ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 55 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:59226 req_id = bb8e5cb1-442f-4eac-91f0-f0d2e61db306 response_code = 200 uri = /baseboard
25792024-12-13T22:38:16.121ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:42393
25802024-12-13T22:38:16.121ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 60 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:42393 req_id = de1d33fa-faf5-4c89-a20b-0f2a946a10b2 response_code = 200 uri = /vpc-routes
25812024-12-13T22:38:16.161ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:57642
25822024-12-13T22:38:16.162ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:57642 req_id = 03395912-0d94-4142-bd84-1203eda51c2f response_code = 200 uri = /v2p
25832024-12-13T22:38:16.164ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
25842024-12-13T22:38:16.164ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 57 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42393 req_id = f9d560d2-dced-44fd-bbeb-9d83db30469a response_code = 204 uri = /eip-gateways
25852024-12-13T22:38:16.174ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 229 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:42393 req_id = aeeee1c8-09a2-43f4-bcaf-27ab0236a6b3 response_code = 204 uri = /vpc-routes
25862024-12-13T22:38:16.238ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:40948
25872024-12-13T22:38:16.239ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
25882024-12-13T22:38:16.284ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
25892024-12-13T22:38:16.359ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 120287 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:40948 req_id = f135ca59-8ee9-49c0-b5e9-252fc89b0b0d response_code = 204 uri = /switch-ports
25902024-12-13T22:38:16.359ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:47010
25912024-12-13T22:38:16.359ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 122 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:47010 req_id = 6047ab90-22e9-4181-a2c1-4c6c51e81412 response_code = 200 uri = /vpc-routes
25922024-12-13T22:38:16.359ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 232 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:40948 req_id = 8c94ef9f-020c-4a55-842b-5e2f2b12c868 response_code = 200 uri = /network-bootstore-config
25932024-12-13T22:38:16.373ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:35943
25942024-12-13T22:38:16.373ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
25952024-12-13T22:38:16.417ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
25962024-12-13T22:38:16.483ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 109927 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:35943 req_id = b689225b-527e-4450-8400-ab3bfe1b3077 response_code = 204 uri = /switch-ports
25972024-12-13T22:38:16.483ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
25982024-12-13T22:38:16.483ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 251 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:47010 req_id = 5f79c880-769b-4b51-a7d6-e2681e26527d response_code = 204 uri = /eip-gateways
25992024-12-13T22:38:16.483ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 339 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:35943 req_id = 83bd5a75-6eed-485a-a016-950448a4a300 response_code = 200 uri = /network-bootstore-config
26002024-12-13T22:38:16.508ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:55303
26012024-12-13T22:38:16.509ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 174 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:55303 req_id = 3a59840f-e80b-4ed5-83b1-5b04c6e7c2f9 response_code = 200 uri = /v2p
26022024-12-13T22:38:16.515ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 312 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:47010 req_id = 4e8a8753-b365-4389-9010-dbf9ebcaaed4 response_code = 204 uri = /vpc-routes
26032024-12-13T22:38:38.198ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 9908 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = 6653f64b-43c4-49da-a7be-9eee1ffddda2 response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
26042024-12-13T22:38:45.077ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60110
26052024-12-13T22:38:45.077ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60110 req_id = c642c10a-515d-4501-91dd-453c7497892e response_code = 200 uri = /vpc-routes
26062024-12-13T22:38:45.204ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:60836
26072024-12-13T22:38:45.204ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 153 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:60836 req_id = 964170a9-6423-420b-890a-4b4c18ea7860 response_code = 200 uri = /v2p
26082024-12-13T22:38:45.246ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
26092024-12-13T22:38:45.246ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:60110 req_id = 622032c9-749b-4b30-99a9-f6da090fb5df response_code = 204 uri = /eip-gateways
26102024-12-13T22:38:45.255ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 246 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:60110 req_id = a06eeafa-fc07-4bed-a667-2b8de08f2265 response_code = 204 uri = /vpc-routes
26112024-12-13T22:38:45.290ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48821
26122024-12-13T22:38:45.290ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
26132024-12-13T22:38:45.330ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
26142024-12-13T22:38:45.386ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96384 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:48821 req_id = f761920c-9aac-4877-b8dd-f8a4c81a88e7 response_code = 204 uri = /switch-ports
26152024-12-13T22:38:45.387ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48821 req_id = d3224e8d-6562-40ea-a61e-6a00df57c458 response_code = 200 uri = /network-bootstore-config
26162024-12-13T22:38:45.684ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:59070
26172024-12-13T22:38:45.684ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:59070 req_id = 60d6bd95-10e8-4ac7-af82-4093bb04cbd1 response_code = 200 uri = /baseboard
26182024-12-13T22:38:46.262ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63883
26192024-12-13T22:38:46.262ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 90 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63883 req_id = dde7c821-5b7f-4c5d-943d-2023f553f862 response_code = 200 uri = /vpc-routes
26202024-12-13T22:38:46.304ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:51265
26212024-12-13T22:38:46.304ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 78 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:51265 req_id = e0e639b6-29bb-49fb-8ab7-48d2a46f817b response_code = 200 uri = /vpc-routes
26222024-12-13T22:38:46.307ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
26232024-12-13T22:38:46.307ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 87 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:63883 req_id = 3c6412f4-b2d4-4d58-b3d5-280f5ba8a6d2 response_code = 204 uri = /eip-gateways
26242024-12-13T22:38:46.315ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 239 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:63883 req_id = c9de36ae-6fb7-402e-9c62-59c82dd3ff0a response_code = 204 uri = /vpc-routes
26252024-12-13T22:38:46.355ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
26262024-12-13T22:38:46.355ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 53 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:51265 req_id = 8e7d3d8d-c3b9-4aa2-abb4-7fc2e7078480 response_code = 204 uri = /eip-gateways
26272024-12-13T22:38:46.384ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:56959
26282024-12-13T22:38:46.384ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 107 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:56959 req_id = 1e754514-6a61-4643-8d34-a2f0b060988f response_code = 200 uri = /v2p
26292024-12-13T22:38:46.387ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 185 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:51265 req_id = b6ef00a1-87aa-4d85-b0da-34b730401093 response_code = 204 uri = /vpc-routes
26302024-12-13T22:38:46.398ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33189
26312024-12-13T22:38:46.398ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
26322024-12-13T22:38:46.440ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
26332024-12-13T22:38:46.498ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99733 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:33189 req_id = 8cbb3deb-116f-4da4-98b0-e6ffa6821911 response_code = 204 uri = /switch-ports
26342024-12-13T22:38:46.498ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59704
26352024-12-13T22:38:46.498ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 206 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59704 req_id = c1b12216-577c-4bdd-856b-52e36f95a7f9 response_code = 200 uri = /v2p
26362024-12-13T22:38:46.498ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 239 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33189 req_id = cc775f75-9bb2-47cd-a44b-4a17f3531afe response_code = 200 uri = /network-bootstore-config
26372024-12-13T22:38:46.653ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50011
26382024-12-13T22:38:46.653ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
26392024-12-13T22:38:46.692ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
26402024-12-13T22:38:46.750ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96891 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:50011 req_id = 0bfc40ae-20e9-4c40-bbdf-99efff736f99 response_code = 204 uri = /switch-ports
26412024-12-13T22:38:46.756ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 82 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50011 req_id = adba1d49-e598-4ef1-ad09-6e27016b33a4 response_code = 200 uri = /network-bootstore-config
26422024-12-13T22:39:08.195ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7300 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = ad4fd8cb-4f8f-4590-9c46-241e5e42111d response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
26432024-12-13T22:39:14.947ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36999
26442024-12-13T22:39:14.947ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36999 req_id = 1fb85993-fe8e-4306-9ec0-e8333688aabd response_code = 200 uri = /vpc-routes
26452024-12-13T22:39:14.988ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44553
26462024-12-13T22:39:14.989ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 125 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44553 req_id = 528e143c-1196-4ffe-acb8-26c3c2fcca51 response_code = 200 uri = /v2p
26472024-12-13T22:39:14.994ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
26482024-12-13T22:39:14.994ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 58 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36999 req_id = 72281ad1-16c6-4f71-a8bf-743ab7cf904f response_code = 204 uri = /eip-gateways
26492024-12-13T22:39:15.005ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 270 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36999 req_id = 04d9578d-4874-49b7-99e4-cdd88b0c08d9 response_code = 204 uri = /vpc-routes
26502024-12-13T22:39:15.020ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:56526
26512024-12-13T22:39:15.020ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
26522024-12-13T22:39:15.062ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
26532024-12-13T22:39:15.121ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101180 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:56526 req_id = ecc8b1c8-2121-4fc6-b6a7-9d0f0d72b7e2 response_code = 204 uri = /switch-ports
26542024-12-13T22:39:15.122ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:56526 req_id = 46a35221-5105-491b-9598-f28bc1e5839d response_code = 200 uri = /network-bootstore-config
26552024-12-13T22:39:15.718ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:44094
26562024-12-13T22:39:15.718ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 66 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:44094 req_id = 23debf79-f723-40b7-8411-abb675f0ff15 response_code = 200 uri = /baseboard
26572024-12-13T22:39:16.171ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:65435
26582024-12-13T22:39:16.171ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:65435 req_id = edfd3ce5-b60b-4a4a-b2fe-11a51d91b2ed response_code = 200 uri = /vpc-routes
26592024-12-13T22:39:16.202ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59849
26602024-12-13T22:39:16.202ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 120 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59849 req_id = 958fa387-746d-4d84-b94c-9098b5ebedd0 response_code = 200 uri = /v2p
26612024-12-13T22:39:16.232ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36414
26622024-12-13T22:39:16.232ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36414 req_id = f565c916-5f87-4e23-93af-5e31f4ca7fbf response_code = 200 uri = /vpc-routes
26632024-12-13T22:39:16.249ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
26642024-12-13T22:39:16.249ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 66 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:65435 req_id = bab569f8-1575-4a45-95f0-d22d5c9ea2e3 response_code = 204 uri = /eip-gateways
26652024-12-13T22:39:16.260ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 263 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:65435 req_id = 75124c0a-322f-45c5-bdb0-9c467b50e01c response_code = 204 uri = /vpc-routes
26662024-12-13T22:39:16.276ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:50780
26672024-12-13T22:39:16.276ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
26682024-12-13T22:39:16.317ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
26692024-12-13T22:39:16.377ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101238 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:50780 req_id = 78b343c7-7c31-4dcb-9340-8ae6ace7be8e response_code = 204 uri = /switch-ports
26702024-12-13T22:39:16.377ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:53269
26712024-12-13T22:39:16.377ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
26722024-12-13T22:39:16.377ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 123 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:36414 req_id = 1e11229e-7dbe-49f0-8468-31b0f564d02d response_code = 204 uri = /eip-gateways
26732024-12-13T22:39:16.377ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34464
26742024-12-13T22:39:16.378ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
26752024-12-13T22:39:16.378ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 296 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:50780 req_id = 132da0c0-3519-4554-bf6d-08451d965f81 response_code = 200 uri = /network-bootstore-config
26762024-12-13T22:39:16.381ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 3501 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34464 req_id = 1723d68e-1bcb-4ce9-b5e8-416686f3b0a9 response_code = 200 uri = /v2p
26772024-12-13T22:39:16.387ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 399 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:36414 req_id = cb31bfa3-5551-4377-ae54-81092f665f57 response_code = 204 uri = /vpc-routes
26782024-12-13T22:39:16.416ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
26792024-12-13T22:39:16.476ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98334 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:53269 req_id = 97b8c854-fd52-4bb5-982d-206329e89e40 response_code = 204 uri = /switch-ports
26802024-12-13T22:39:16.476ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 159 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:53269 req_id = 98b185a6-698d-4221-b1ca-e2312883d0d1 response_code = 200 uri = /network-bootstore-config
26812024-12-13T22:39:38.195ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7243 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = 4e171dab-94be-4569-9fa7-df34b3d99610 response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
26822024-12-13T22:39:45.042ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:35784
26832024-12-13T22:39:45.042ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 91 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:35784 req_id = 15993503-d194-4613-9dad-011f2349bf7b response_code = 200 uri = /vpc-routes
26842024-12-13T22:39:45.129ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
26852024-12-13T22:39:45.129ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35784 req_id = 0fe51a7e-e3cb-43aa-ad65-c84a01e0b13a response_code = 204 uri = /eip-gateways
26862024-12-13T22:39:45.142ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 245 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:35784 req_id = d60546da-796c-4c8e-bf1f-7083e096e3d8 response_code = 204 uri = /vpc-routes
26872024-12-13T22:39:45.308ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41784
26882024-12-13T22:39:45.308ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 136 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41784 req_id = 75433b50-1409-4762-85dd-74b6fa391fe2 response_code = 200 uri = /v2p
26892024-12-13T22:39:45.324ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:59782
26902024-12-13T22:39:45.324ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
26912024-12-13T22:39:45.368ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
26922024-12-13T22:39:45.426ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 101883 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:59782 req_id = 4e19d144-72dd-4d07-ab68-9eb70444966c response_code = 204 uri = /switch-ports
26932024-12-13T22:39:45.426ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36693
26942024-12-13T22:39:45.426ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 169 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:59782 req_id = 6670ffc6-f753-4915-a4a3-b7bcddc9a5d4 response_code = 200 uri = /network-bootstore-config
26952024-12-13T22:39:45.426ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
26962024-12-13T22:39:45.426ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
26972024-12-13T22:39:45.426ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:177
26982024-12-13T22:39:45.427ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:177
26992024-12-13T22:39:45.442ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27002024-12-13T22:39:45.442ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5
27012024-12-13T22:39:45.463ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44569
27022024-12-13T22:39:45.464ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0e0028b4-c96e-4259-b0d9-2dd1e967b57e, kind: Service { id: 3e0d656e-3552-43ff-8e95-5ed82471e0ae }, name: Name("external-dns-3e0d656e-3552-43ff-8e95-5ed82471e0ae"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 156])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9fbc6464-5658-4709-85f5-cba4ad21c9de, kind: Service { id: 0ef774ca-06b2-40a8-a632-7304d594c07a }, name: Name("external-dns-0ef774ca-06b2-40a8-a632-7304d594c07a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 39])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: bb1ce312-163f-477c-9052-27f110965501, kind: Service { id: ef47bd05-ad7e-42a3-8257-dec06acd2b93 }, name: Name("nexus-ef47bd05-ad7e-42a3-8257-dec06acd2b93"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 50])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: be9ea2e0-22c4-425c-9afb-1597ead9d56b, kind: Service { id: b2389da8-cde2-4174-a8e1-a4dabd361b80 }, name: Name("nexus-b2389da8-cde2-4174-a8e1-a4dabd361b80"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 223, 94])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e47aafa9-7219-497b-8e24-b813038ee602, kind: Service { id: 7e3d78a7-b6c3-4636-b217-b0a79485c981 }, name: Name("nexus-7e3d78a7-b6c3-4636-b217-b0a79485c981"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 163])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
27032024-12-13T22:39:45.464ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27042024-12-13T22:39:45.464ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
27052024-12-13T22:39:45.464ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27062024-12-13T22:39:45.464ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27072024-12-13T22:39:45.464ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27082024-12-13T22:39:45.464ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27092024-12-13T22:39:45.464ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 720 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44569 req_id = d470004e-3655-49d1-a7cb-f95e7d8ff443 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
27102024-12-13T22:39:45.470ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27112024-12-13T22:39:45.470ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f
27122024-12-13T22:39:45.495ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27132024-12-13T22:39:45.495ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_9e30dd3c-3e72-4b01-8f5f-662db99dc37a
27142024-12-13T22:39:45.517ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27152024-12-13T22:39:45.517ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8159e296-b0fc-4958-9b6f-9386d6e003fa
27162024-12-13T22:39:45.540ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27172024-12-13T22:39:45.540ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_fb594314-d689-4df6-8176-e09675ab8f03
27182024-12-13T22:39:45.561ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27192024-12-13T22:39:45.562ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_5bc1d750-0ca6-4a08-bf34-a01bf34cccef
27202024-12-13T22:39:45.584ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27212024-12-13T22:39:45.584ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_b80657b8-dd44-496f-8b60-edeffe6ea0c4
27222024-12-13T22:39:45.607ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27232024-12-13T22:39:45.607ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_35c96663-c574-4d07-a692-3eb8a2a79302
27242024-12-13T22:39:45.631ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27252024-12-13T22:39:45.631ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_ffd405c4-ad68-4401-9d01-3a9d70b7775f
27262024-12-13T22:39:45.644ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 218049 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36693 req_id = 5e726d76-5f38-49dd-b926-d426f0d05db7 response_code = 200 uri = /inventory
27272024-12-13T22:39:45.752ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:63339
27282024-12-13T22:39:45.752ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 74 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:63339 req_id = 65428682-a0b0-41ee-9d0a-a75ca91df7ae response_code = 200 uri = /baseboard
27292024-12-13T22:39:46.317ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:39015
27302024-12-13T22:39:46.317ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 65 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:39015 req_id = 4080483e-4237-41e9-be20-3e4b35b7551a response_code = 200 uri = /vpc-routes
27312024-12-13T22:39:46.332ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38723
27322024-12-13T22:39:46.332ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 47 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38723 req_id = 35cd0711-4294-4234-a203-8b1ac1c9af74 response_code = 200 uri = /vpc-routes
27332024-12-13T22:39:46.380ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
27342024-12-13T22:39:46.380ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:38723 req_id = d3c6693c-5483-4c71-9fec-511cd473e1d2 response_code = 204 uri = /eip-gateways
27352024-12-13T22:39:46.423ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:38405
27362024-12-13T22:39:46.424ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 105 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:38405 req_id = aa588c37-f16a-4fb2-bc02-215d70cfc3e3 response_code = 200 uri = /v2p
27372024-12-13T22:39:46.437ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
27382024-12-13T22:39:46.437ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 43 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:39015 req_id = 19f7711e-d237-4aef-8d08-154d5428bd3f response_code = 204 uri = /eip-gateways
27392024-12-13T22:39:46.446ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 214 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:39015 req_id = fe839314-9f9d-4122-a290-beccdb357519 response_code = 204 uri = /vpc-routes
27402024-12-13T22:39:46.447ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:63607
27412024-12-13T22:39:46.447ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
27422024-12-13T22:39:46.447ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
27432024-12-13T22:39:46.448ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:177
27442024-12-13T22:39:46.448ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53583
27452024-12-13T22:39:46.448ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53583 req_id = faea3e64-aec3-482d-a468-9bfcd75740b5 response_code = 200 uri = /v2p
27462024-12-13T22:39:46.448ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:177
27472024-12-13T22:39:46.457ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 428 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:38723 req_id = 5e35887b-68ec-4cfd-a7ad-557652d60362 response_code = 204 uri = /vpc-routes
27482024-12-13T22:39:46.461ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27492024-12-13T22:39:46.461ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5
27502024-12-13T22:39:46.493ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27512024-12-13T22:39:46.493ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f
27522024-12-13T22:39:46.498ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:43761
27532024-12-13T22:39:46.499ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
27542024-12-13T22:39:46.524ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27552024-12-13T22:39:46.524ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_9e30dd3c-3e72-4b01-8f5f-662db99dc37a
27562024-12-13T22:39:46.545ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
27572024-12-13T22:39:46.554ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27582024-12-13T22:39:46.554ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8159e296-b0fc-4958-9b6f-9386d6e003fa
27592024-12-13T22:39:46.582ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27602024-12-13T22:39:46.583ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_fb594314-d689-4df6-8176-e09675ab8f03
27612024-12-13T22:39:46.603ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:42784
27622024-12-13T22:39:46.607ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
27632024-12-13T22:39:46.607ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
27642024-12-13T22:39:46.607ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/config/omicron-zones.json file = common/src/ledger.rs:177
27652024-12-13T22:39:46.608ZINFOSledAgent (ServiceManager): Reading ledger from /pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/config/omicron-zones.json file = common/src/ledger.rs:177
27662024-12-13T22:39:46.612ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27672024-12-13T22:39:46.612ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_5bc1d750-0ca6-4a08-bf34-a01bf34cccef
27682024-12-13T22:39:46.613ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53514
27692024-12-13T22:39:46.619ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 119982 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:43761 req_id = b757bca4-1d2e-4b4e-a828-1d9a15836442 response_code = 204 uri = /switch-ports
27702024-12-13T22:39:46.619ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 297 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:43761 req_id = 89f9591c-4383-4856-ac3e-6fdfbace922f response_code = 200 uri = /network-bootstore-config
27712024-12-13T22:39:46.623ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0e0028b4-c96e-4259-b0d9-2dd1e967b57e, kind: Service { id: 3e0d656e-3552-43ff-8e95-5ed82471e0ae }, name: Name("external-dns-3e0d656e-3552-43ff-8e95-5ed82471e0ae"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 156])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9fbc6464-5658-4709-85f5-cba4ad21c9de, kind: Service { id: 0ef774ca-06b2-40a8-a632-7304d594c07a }, name: Name("external-dns-0ef774ca-06b2-40a8-a632-7304d594c07a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 39])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: bb1ce312-163f-477c-9052-27f110965501, kind: Service { id: ef47bd05-ad7e-42a3-8257-dec06acd2b93 }, name: Name("nexus-ef47bd05-ad7e-42a3-8257-dec06acd2b93"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 50])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: be9ea2e0-22c4-425c-9afb-1597ead9d56b, kind: Service { id: b2389da8-cde2-4174-a8e1-a4dabd361b80 }, name: Name("nexus-b2389da8-cde2-4174-a8e1-a4dabd361b80"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 223, 94])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e47aafa9-7219-497b-8e24-b813038ee602, kind: Service { id: 7e3d78a7-b6c3-4636-b217-b0a79485c981 }, name: Name("nexus-7e3d78a7-b6c3-4636-b217-b0a79485c981"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 163])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
27722024-12-13T22:39:46.623ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27732024-12-13T22:39:46.623ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
27742024-12-13T22:39:46.623ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27752024-12-13T22:39:46.623ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27762024-12-13T22:39:46.623ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27772024-12-13T22:39:46.624ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27782024-12-13T22:39:46.624ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4811 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:53514 req_id = f21b8502-57a2-4633-ae00-1faa5312e6c9 response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
27792024-12-13T22:39:46.629ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: d3dcd055-d94b-4283-a632-f7eab7bbead5 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27802024-12-13T22:39:46.629ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_d3dcd055-d94b-4283-a632-f7eab7bbead5
27812024-12-13T22:39:46.645ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27822024-12-13T22:39:46.645ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_b80657b8-dd44-496f-8b60-edeffe6ea0c4
27832024-12-13T22:39:46.651ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:63007
27842024-12-13T22:39:46.653ZINFOSledAgent (PortManager): Ensuring VPC firewall rules file = illumos-utils/src/opte/port_manager.rs:767 rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 0e0028b4-c96e-4259-b0d9-2dd1e967b57e, kind: Service { id: 3e0d656e-3552-43ff-8e95-5ed82471e0ae }, name: Name("external-dns-3e0d656e-3552-43ff-8e95-5ed82471e0ae"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 167, 156])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 9fbc6464-5658-4709-85f5-cba4ad21c9de, kind: Service { id: 0ef774ca-06b2-40a8-a632-7304d594c07a }, name: Name("external-dns-0ef774ca-06b2-40a8-a632-7304d594c07a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 198, 39])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: bb1ce312-163f-477c-9052-27f110965501, kind: Service { id: ef47bd05-ad7e-42a3-8257-dec06acd2b93 }, name: Name("nexus-ef47bd05-ad7e-42a3-8257-dec06acd2b93"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 166, 50])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: be9ea2e0-22c4-425c-9afb-1597ead9d56b, kind: Service { id: b2389da8-cde2-4174-a8e1-a4dabd361b80 }, name: Name("nexus-b2389da8-cde2-4174-a8e1-a4dabd361b80"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 223, 94])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: e47aafa9-7219-497b-8e24-b813038ee602, kind: Service { id: 7e3d78a7-b6c3-4636-b217-b0a79485c981 }, name: Name("nexus-7e3d78a7-b6c3-4636-b217-b0a79485c981"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 177, 163])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }] vni = Vni(100)
27852024-12-13T22:39:46.653ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte3" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27862024-12-13T22:39:46.653ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte0" rules = []
27872024-12-13T22:39:46.653ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte5" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(UDP), ports: PortList([53]) }, action: Allow, priority: 65534 }]
27882024-12-13T22:39:46.653ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte1" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27892024-12-13T22:39:46.653ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte4" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27902024-12-13T22:39:46.653ZINFOSledAgent (PortManager): Setting OPTE firewall rules file = illumos-utils/src/opte/port_manager.rs:786 port = "opte2" rules = [FirewallRule { direction: In, filters: Filters { hosts: Any, protocol: Proto(TCP), ports: PortList([80, 443]) }, action: Allow, priority: 65534 }]
27912024-12-13T22:39:46.653ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 670 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:63007 req_id = 2ba47358-0a14-440c-804b-03471f9a677d response_code = 204 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
27922024-12-13T22:39:46.662ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 0d689a16-0a28-4b27-af5a-3b4bf99f365f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27932024-12-13T22:39:46.662ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_0d689a16-0a28-4b27-af5a-3b4bf99f365f
27942024-12-13T22:39:46.674ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:34061
27952024-12-13T22:39:46.674ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
27962024-12-13T22:39:46.690ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 9e30dd3c-3e72-4b01-8f5f-662db99dc37a (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27972024-12-13T22:39:46.690ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_9e30dd3c-3e72-4b01-8f5f-662db99dc37a
27982024-12-13T22:39:46.704ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
27992024-12-13T22:39:46.704ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_35c96663-c574-4d07-a692-3eb8a2a79302
28002024-12-13T22:39:46.713ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
28012024-12-13T22:39:46.721ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 8159e296-b0fc-4958-9b6f-9386d6e003fa (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28022024-12-13T22:39:46.721ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_8159e296-b0fc-4958-9b6f-9386d6e003fa
28032024-12-13T22:39:46.735ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28042024-12-13T22:39:46.735ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_ffd405c4-ad68-4401-9d01-3a9d70b7775f
28052024-12-13T22:39:46.757ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 309178 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:63607 req_id = 7030b521-d02e-4ca5-acb8-d51dad82de35 response_code = 200 uri = /inventory
28062024-12-13T22:39:46.759ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: fb594314-d689-4df6-8176-e09675ab8f03 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28072024-12-13T22:39:46.759ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_fb594314-d689-4df6-8176-e09675ab8f03
28082024-12-13T22:39:46.778ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103932 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:34061 req_id = ce9cfa99-558e-4fc0-9105-478bf9f68318 response_code = 204 uri = /switch-ports
28092024-12-13T22:39:46.778ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 159 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:34061 req_id = d7deea9d-dc60-4ca6-af68-dab492d68ed2 response_code = 200 uri = /network-bootstore-config
28102024-12-13T22:39:46.785ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 5bc1d750-0ca6-4a08-bf34-a01bf34cccef (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28112024-12-13T22:39:46.785ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_5bc1d750-0ca6-4a08-bf34-a01bf34cccef
28122024-12-13T22:39:46.809ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: b80657b8-dd44-496f-8b60-edeffe6ea0c4 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28132024-12-13T22:39:46.809ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_b80657b8-dd44-496f-8b60-edeffe6ea0c4
28142024-12-13T22:39:46.833ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: 35c96663-c574-4d07-a692-3eb8a2a79302 (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28152024-12-13T22:39:46.833ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_35c96663-c574-4d07-a692-3eb8a2a79302
28162024-12-13T22:39:46.856ZINFOSledAgent (StorageManager): Received DatasetsList { zpool: ZpoolName { id: ffd405c4-ad68-4401-9d01-3a9d70b7775f (zpool), kind: External }, tx: ... } file = sled-storage/src/manager.rs:603
28172024-12-13T22:39:46.857ZINFOSledAgent (StorageManager): Listing datasets within zpool file = sled-storage/src/manager.rs:1019 request = datasets_list zpool = oxp_ffd405c4-ad68-4401-9d01-3a9d70b7775f
28182024-12-13T22:39:46.870ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 263415 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:42784 req_id = 4e6fd652-a14d-48a5-af80-2324141c0e02 response_code = 200 uri = /inventory
28192024-12-13T22:40:08.195ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8698 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = ed64a99b-6847-4d6e-b8bf-08cf4bb6d45f response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
28202024-12-13T22:40:14.811ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48025
28212024-12-13T22:40:14.811ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48025 req_id = 84764f96-f369-466d-8727-94be5b276e55 response_code = 200 uri = /vpc-routes
28222024-12-13T22:40:14.869ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
28232024-12-13T22:40:14.869ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 94 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48025 req_id = eef6dd1f-e649-41a1-9811-9108715ae997 response_code = 204 uri = /eip-gateways
28242024-12-13T22:40:14.878ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 255 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48025 req_id = 935e0544-2805-4266-a1da-2c8779086f30 response_code = 204 uri = /vpc-routes
28252024-12-13T22:40:15.033ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41904
28262024-12-13T22:40:15.033ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
28272024-12-13T22:40:15.075ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
28282024-12-13T22:40:15.134ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 100564 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:41904 req_id = 276aa341-883b-441a-8fad-9354ebae3769 response_code = 204 uri = /switch-ports
28292024-12-13T22:40:15.134ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44930
28302024-12-13T22:40:15.134ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 178 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:44930 req_id = af771629-f304-4724-8756-4322ef53f8f1 response_code = 200 uri = /v2p
28312024-12-13T22:40:15.134ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 212 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41904 req_id = fa68cb82-5500-4c72-b374-103af43764ef response_code = 200 uri = /network-bootstore-config
28322024-12-13T22:40:15.786ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:32776
28332024-12-13T22:40:15.786ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 67 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:32776 req_id = 6a0ba642-f18e-4ce7-adf7-5328a030a7ae response_code = 200 uri = /baseboard
28342024-12-13T22:40:16.058ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:45866
28352024-12-13T22:40:16.058ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 59 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:45866 req_id = 3ade4fb4-1fdd-4812-9ddd-20e0b1e4af0b response_code = 200 uri = /vpc-routes
28362024-12-13T22:40:16.136ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
28372024-12-13T22:40:16.136ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 99 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45866 req_id = e12825b0-d393-42e7-920f-9ec7d489db55 response_code = 204 uri = /eip-gateways
28382024-12-13T22:40:16.145ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 196 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:45866 req_id = 018e3667-275e-4c3d-b613-824e40bef2b4 response_code = 204 uri = /vpc-routes
28392024-12-13T22:40:16.213ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:43951
28402024-12-13T22:40:16.213ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 45 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:43951 req_id = f08e0b1f-3334-4499-bcb7-16990f2f434c response_code = 200 uri = /vpc-routes
28412024-12-13T22:40:16.230ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51833
28422024-12-13T22:40:16.230ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51833 req_id = 86869cea-ee11-42f5-b42d-32f425aa7aaf response_code = 200 uri = /v2p
28432024-12-13T22:40:16.239ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:55662
28442024-12-13T22:40:16.239ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
28452024-12-13T22:40:16.279ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
28462024-12-13T22:40:16.338ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98345 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:55662 req_id = d32eb2bc-1471-4d11-a82c-9c5bf4a9dace response_code = 204 uri = /switch-ports
28472024-12-13T22:40:16.338ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
28482024-12-13T22:40:16.338ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 202 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43951 req_id = 320f8469-2581-4090-8bb0-1604180efb76 response_code = 204 uri = /eip-gateways
28492024-12-13T22:40:16.338ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 313 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:55662 req_id = bb9584ce-0617-497e-8150-ea2cff3aa08b response_code = 200 uri = /network-bootstore-config
28502024-12-13T22:40:16.348ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46575
28512024-12-13T22:40:16.348ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 383 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:43951 req_id = 6ef1360a-68ea-4a82-9a81-deafd3e001d1 response_code = 204 uri = /vpc-routes
28522024-12-13T22:40:16.348ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 263 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46575 req_id = 873bfe12-3835-4c4d-a9e2-d029dc57e889 response_code = 200 uri = /v2p
28532024-12-13T22:40:16.389ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46116
28542024-12-13T22:40:16.389ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
28552024-12-13T22:40:16.428ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
28562024-12-13T22:40:16.484ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95247 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:46116 req_id = 171eed38-8c98-4fb5-8d45-654d2cf3c0ef response_code = 204 uri = /switch-ports
28572024-12-13T22:40:16.485ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 233 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46116 req_id = 79e64561-dc00-4b08-85fa-849632253bbf response_code = 200 uri = /network-bootstore-config
28582024-12-13T22:40:38.221ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 12600 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = dda15e02-3d4e-43f0-93c9-2a7956cee77b response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
28592024-12-13T22:40:45.035ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45406
28602024-12-13T22:40:45.035ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 76 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45406 req_id = 96d188c7-38a2-47a6-acb1-da36fc9c41cb response_code = 200 uri = /vpc-routes
28612024-12-13T22:40:45.054ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
28622024-12-13T22:40:45.054ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 81 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45406 req_id = ee2627e3-27a7-4a3a-a27f-abbb6317f7eb response_code = 204 uri = /eip-gateways
28632024-12-13T22:40:45.063ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 229 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:45406 req_id = f775111a-d72f-43a8-90ed-3c92fadd03ce response_code = 204 uri = /vpc-routes
28642024-12-13T22:40:45.211ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:45475
28652024-12-13T22:40:45.211ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 110 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:45475 req_id = a7f11cd6-d164-4cb2-b631-e1e38b50e47e response_code = 200 uri = /v2p
28662024-12-13T22:40:45.315ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:41758
28672024-12-13T22:40:45.315ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
28682024-12-13T22:40:45.357ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
28692024-12-13T22:40:45.412ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 96841 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:41758 req_id = e25411e9-73c3-4fd3-8ac5-9a701eea607e response_code = 204 uri = /switch-ports
28702024-12-13T22:40:45.412ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:41758 req_id = 8056010a-ec6d-482e-ad09-cb1ea6fba3fb response_code = 200 uri = /network-bootstore-config
28712024-12-13T22:40:45.820ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:34578
28722024-12-13T22:40:45.821ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 71 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:34578 req_id = 28b0ae1f-c44e-4255-b75b-3d610b774311 response_code = 200 uri = /baseboard
28732024-12-13T22:40:46.255ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:59415
28742024-12-13T22:40:46.256ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:59415 req_id = 0ed00f97-3bee-444f-a04f-ef11db77666d response_code = 200 uri = /vpc-routes
28752024-12-13T22:40:46.272ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:61226
28762024-12-13T22:40:46.272ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 132 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:61226 req_id = 8121e4e7-3af7-418b-a6c2-1faa8317175d response_code = 200 uri = /v2p
28772024-12-13T22:40:46.297ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
28782024-12-13T22:40:46.297ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59415 req_id = f01e6ba0-e090-4524-8411-66f28a094f7a response_code = 204 uri = /eip-gateways
28792024-12-13T22:40:46.320ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:36669
28802024-12-13T22:40:46.321ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 38 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:36669 req_id = f4085448-5a17-4737-8034-ccc448727149 response_code = 200 uri = /vpc-routes
28812024-12-13T22:40:46.335ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
28822024-12-13T22:40:46.335ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 41 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:36669 req_id = bf60e3e6-691d-43c2-9e1b-8acbf889865a response_code = 204 uri = /eip-gateways
28832024-12-13T22:40:46.336ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:53953
28842024-12-13T22:40:46.336ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:53953 req_id = 3eeb9a13-2b1a-42c5-9766-12fd669ae689 response_code = 200 uri = /v2p
28852024-12-13T22:40:46.344ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 265 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:36669 req_id = 784c44dc-5fe6-4ac8-b6dd-dc06e70cefdc response_code = 204 uri = /vpc-routes
28862024-12-13T22:40:46.345ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 164 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:59415 req_id = ee2c1008-a873-4b67-802e-8129ef2b7862 response_code = 204 uri = /vpc-routes
28872024-12-13T22:40:46.423ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:62322
28882024-12-13T22:40:46.423ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
28892024-12-13T22:40:46.462ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
28902024-12-13T22:40:46.515ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 92146 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:62322 req_id = 3a72fe60-3d83-40c4-8b35-4c73b81daeea response_code = 204 uri = /switch-ports
28912024-12-13T22:40:46.516ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 103 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:62322 req_id = 793f17cb-c522-4ef0-b22c-ec6c200e6348 response_code = 200 uri = /network-bootstore-config
28922024-12-13T22:40:46.556ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:33681
28932024-12-13T22:40:46.556ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
28942024-12-13T22:40:46.593ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
28952024-12-13T22:40:46.652ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 95873 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:33681 req_id = ea431579-b3b9-4798-95ce-94b86bfbed2f response_code = 204 uri = /switch-ports
28962024-12-13T22:40:46.653ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:33681 req_id = e1c33ed0-5237-4946-a67a-ba1c1ccb62d1 response_code = 200 uri = /network-bootstore-config
28972024-12-13T22:41:08.195ZINFOSledAgent (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 8411 local_addr = [fd00:1122:3344:101::1]:62120 method = GET remote_addr = [fd00:1122:3344:101::d]:38161 req_id = 0bb2bd54-3225-4b25-984f-f78601cacbdb response_code = 200 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 uri = /35eb16d3-adec-4806-a15c-ab7a15d18765
28982024-12-13T22:41:09.735ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:36825
28992024-12-13T22:41:09.735ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 163 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:36825 req_id = a6055181-7583-40f4-ae41-cbc31cf5af47 response_code = 200 uri = /v2p
29002024-12-13T22:41:09.736ZINFOSledAgent (PortManager): Mapping virtual NIC to physical host file = illumos-utils/src/opte/port_manager.rs:874 mapping = VirtualNetworkInterfaceHost { virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), physical_host_ip: fd00:1122:3344:101::1, vni: Vni(14180905) }
29012024-12-13T22:41:09.736ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 589 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:36825 req_id = f1e81c48-40b2-4d47-b725-47fb1337711b response_code = 204 uri = /v2p
29022024-12-13T22:41:09.837ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:44974
29032024-12-13T22:41:09.839ZINFOSledAgent (InstanceManager): ensuring instance is registered file = sled-agent/src/instance_manager.rs:555 hardware = InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: b9fc0549-4550-46e9-aa06-4737b874563c, kind: Instance { id: ae4fa161-b162-4506-a738-be9e4cbbf9c6 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(14180905), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: b9fc0549-4550-46e9-aa06-4737b874563c, kind: Instance { id: ae4fa161-b162-4506-a738-be9e4cbbf9c6 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(14180905), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: b9fc0549-4550-46e9-aa06-4737b874563c, kind: Instance { id: ae4fa161-b162-4506-a738-be9e4cbbf9c6 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(14180905), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(14180905))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: b9fc0549-4550-46e9-aa06-4737b874563c, kind: Instance { id: ae4fa161-b162-4506-a738-be9e4cbbf9c6 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(14180905), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-c5d36bcf34c", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 06e1ccd5-ea5b-453d-b224-e431e78bf6e2, read_only_parent: Some(Volume { block_size: 512, id: fb51480e-bf05-453a-ac5a-3d2baabe8673, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 2f1461d9-16fd-4eb2-b628-e7359780cd1c, key: Some("jq+eovHEbhFnQU7jBKueg45vd1Jp93kigfkTsDg2jrU="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::13]:19002", "[fd00:1122:3344:101::18]:19001", "[fd00:1122:3344:101::17]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 06e1ccd5-ea5b-453d-b224-e431e78bf6e2, key: Some("pFGlj/HhngBjBZ456V2a8y7l+b86nknAJfkEG0D+BfI="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::18]:19002", "[fd00:1122:3344:101::14]:19000", "[fd00:1122:3344:101::1a]:19000"] } }] } }], boot_settings: Some(BootSettings { order: [BootOrderEntry { name: "disk-c5d36bcf34c" }] }), cloud_init_bytes: Some(..) } instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 metadata = InstanceMetadata { silo_id: 2af64f21-336d-4479-8a3e-83cad15fcca8, project_id: d713b47a-c064-4a76-9b87-6bcf34ef199c } migration_id = None propolis_addr = [fd00:1122:3344:101::1:0]:12400 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf vmm_runtime = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-12-13T22:41:09.834117922Z }
29042024-12-13T22:41:09.839ZINFOSledAgent (InstanceManager): registering new instance file = sled-agent/src/instance_manager.rs:588 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 migration_id = None propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
29052024-12-13T22:41:09.839ZINFOSledAgent (InstanceManager): initializing new Instance file = sled-agent/src/instance.rs:1281 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 migration_id = None propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf state = InstanceInitialState { hardware: InstanceHardware { properties: InstanceProperties { ncpus: InstanceCpuCount(2), memory: ByteCount(1073741824), hostname: Hostname("localshark") }, nics: [NetworkInterface { id: b9fc0549-4550-46e9-aa06-4737b874563c, kind: Instance { id: ae4fa161-b162-4506-a738-be9e4cbbf9c6 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(14180905), primary: true, slot: 0, transit_ips: [] }], source_nat: SourceNatConfig { ip: 10.151.2.175, first_port: 0, last_port: 16383 }, ephemeral_ip: Some(10.151.2.176), floating_ips: [], firewall_rules: [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: b9fc0549-4550-46e9-aa06-4737b874563c, kind: Instance { id: ae4fa161-b162-4506-a738-be9e4cbbf9c6 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(14180905), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: b9fc0549-4550-46e9-aa06-4737b874563c, kind: Instance { id: ae4fa161-b162-4506-a738-be9e4cbbf9c6 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(14180905), primary: true, slot: 0, transit_ips: [] }], filter_hosts: Some([Vpc(Vni(14180905))]), filter_ports: None, filter_protocols: None, action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: b9fc0549-4550-46e9-aa06-4737b874563c, kind: Instance { id: ae4fa161-b162-4506-a738-be9e4cbbf9c6 }, name: Name("net0"), ip: 172.30.0.5, mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), vni: Vni(14180905), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(22), last: L4Port(22) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }], dhcp_config: DhcpConfig { dns_servers: [1.1.1.1, 9.9.9.9], host_domain: None, search_domains: [] }, disks: [DiskRequest { device: "nvme", name: "disk-c5d36bcf34c", read_only: false, slot: Slot(0), volume_construction_request: Volume { block_size: 512, id: 06e1ccd5-ea5b-453d-b224-e431e78bf6e2, read_only_parent: Some(Volume { block_size: 512, id: fb51480e-bf05-453a-ac5a-3d2baabe8673, read_only_parent: None, sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 2, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 2f1461d9-16fd-4eb2-b628-e7359780cd1c, key: Some("jq+eovHEbhFnQU7jBKueg45vd1Jp93kigfkTsDg2jrU="), key_pem: None, lossy: false, read_only: true, root_cert_pem: None, target: ["[fd00:1122:3344:101::13]:19002", "[fd00:1122:3344:101::18]:19001", "[fd00:1122:3344:101::17]:19001"] } }] }), sub_volumes: [Region { block_size: 512, blocks_per_extent: 131072, extent_count: 32, gen: 1, opts: CrucibleOpts { cert_pem: None, control: None, flush_timeout: None, id: 06e1ccd5-ea5b-453d-b224-e431e78bf6e2, key: Some("pFGlj/HhngBjBZ456V2a8y7l+b86nknAJfkEG0D+BfI="), key_pem: None, lossy: false, read_only: false, root_cert_pem: None, target: ["[fd00:1122:3344:101::18]:19002", "[fd00:1122:3344:101::14]:19000", "[fd00:1122:3344:101::1a]:19000"] } }] } }], boot_settings: Some(BootSettings { order: [BootOrderEntry { name: "disk-c5d36bcf34c" }] }), cloud_init_bytes: Some(..) }, vmm_runtime: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2024-12-13T22:41:09.834117922Z }, propolis_addr: [fd00:1122:3344:101::1:0]:12400, migration_id: None }
29062024-12-13T22:41:09.839ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2514 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:44974 req_id = 336cdc7f-a308-42ec-a634-c7c18c229fb9 response_code = 200 uri = /vmms/e491dee3-b7b4-4af4-a879-4c884a189dbf
29072024-12-13T22:41:09.922ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37497
29082024-12-13T22:41:09.922ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 309 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:37497 req_id = 1dc20b41-a7ac-4893-8168-63d841424a6d response_code = 200 uri = /vmms/e491dee3-b7b4-4af4-a879-4c884a189dbf/state
29092024-12-13T22:41:09.927ZINFOSledAgent (PortManager): Created OPTE port file = illumos-utils/src/opte/port_manager.rs:447 port = Port { inner: PortInner(PortData { name: "opte6", ip: 172.30.0.5, mac: MacAddr6([168, 64, 37, 240, 0, 0]), slot: 0, vni: Vni { inner: 14180905 }, subnet: V4(Ipv4Net { addr: 172.30.0.0, width: 22 }), gateway: Gateway { mac: MacAddr6([168, 64, 37, 255, 119, 119]), ip: 172.30.0.1 } }) }
29102024-12-13T22:41:09.928ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
29112024-12-13T22:41:09.968ZINFOSledAgent (InstanceManager): Configuring new Omicron zone: oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf file = illumos-utils/src/zone.rs:325 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
29122024-12-13T22:41:09.992ZINFOSledAgent (InstanceManager): Installing Omicron zone: oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf file = illumos-utils/src/zone.rs:363 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
29132024-12-13T22:41:10.082ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:55205
29142024-12-13T22:41:10.082ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 112 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:55205 req_id = 75c382b1-44f1-409d-87c0-ca4cea4848a4 response_code = 200 uri = /vpc-routes
29152024-12-13T22:41:10.104ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {b9fc0549-4550-46e9-aa06-4737b874563c: {10.151.2.176: {e2f6a8ee-b12d-4bea-ac9e-60aff07a0518}, 10.151.2.175: {e2f6a8ee-b12d-4bea-ac9e-60aff07a0518}}} }
29162024-12-13T22:41:11.532ZINFOSledAgent (InstanceManager): Profile for oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf: <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1"> <service_bundle type="profile" name="omicron"> <service version="1" type="service" name="system/illumos/propolis-server"> <instance enabled="true" name="default"> <property_group type="application" name="config"> <propval type="astring" name="datalink" value='oxControlInstance0'/> <propval type="astring" name="gateway" value='fd00:1122:3344:101::1'/> <propval type="astring" name="listen_addr" value='fd00:1122:3344:101::1:0'/> <propval type="astring" name="listen_port" value='12400'/> <propval type="astring" name="metric_addr" value='dns'/> </property_group> </instance> </service> </service_bundle> file = sled-agent/src/profile.rs:34 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
29172024-12-13T22:41:11.532ZINFOSledAgent (InstanceManager): Booting oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf zone file = illumos-utils/src/running_zone.rs:488 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf zone = oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf
29182024-12-13T22:41:11.728ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf") failed: Property not found. retry in 55.423482ms file = illumos-utils/src/svc.rs:36 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf zone = oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf
29192024-12-13T22:41:11.795ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf") failed: Property not found. retry in 94.125347ms file = illumos-utils/src/svc.rs:36 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf zone = oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf
29202024-12-13T22:41:11.904ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf") failed: Property not found. retry in 113.356745ms file = illumos-utils/src/svc.rs:36 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf zone = oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf
29212024-12-13T22:41:12.032ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf") failed: Property not found. retry in 301.393325ms file = illumos-utils/src/svc.rs:36 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf zone = oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf
29222024-12-13T22:41:12.345ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf") failed: Property not found. retry in 1.028636401s file = illumos-utils/src/svc.rs:36 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf zone = oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf
29232024-12-13T22:41:13.386ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf") failed: Property not found. retry in 686.040424ms file = illumos-utils/src/svc.rs:36 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf zone = oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf
29242024-12-13T22:41:14.084ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf") failed: Property not found. retry in 792.123046ms file = illumos-utils/src/svc.rs:36 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf zone = oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf
29252024-12-13T22:41:14.890ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf") failed: Property not found. retry in 1.369354288s file = illumos-utils/src/svc.rs:36 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf zone = oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf
29262024-12-13T22:41:14.946ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:62006
29272024-12-13T22:41:14.960ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:52597
29282024-12-13T22:41:14.960ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 179 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:52597 req_id = 7282ac9a-951a-4a18-a810-09d2831175de response_code = 200 uri = /v2p
29292024-12-13T22:41:15.114ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:40607
29302024-12-13T22:41:15.114ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
29312024-12-13T22:41:15.155ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
29322024-12-13T22:41:15.213ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 98221 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::a]:40607 req_id = a8dc02ec-31d8-43c0-a4f8-20075a31c321 response_code = 204 uri = /switch-ports
29332024-12-13T22:41:15.213ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:40607 req_id = 47ef9a38-f278-4c44-b0de-eda58f64ecf1 response_code = 200 uri = /network-bootstore-config
29342024-12-13T22:41:15.854ZINFOSledAgent (dropshot (BootstrapAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 remote_addr = [fdb0:18c0:4d0d:a02a::2]:46837
29352024-12-13T22:41:15.854ZINFOSledAgent (dropshot (BootstrapAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 83 local_addr = [fdb0:18c0:4d0d:a02a::1]:80 method = GET remote_addr = [fdb0:18c0:4d0d:a02a::2]:46837 req_id = 5d01d012-44f1-4e46-81f2-090963f15e7c response_code = 200 uri = /baseboard
29362024-12-13T22:41:16.021ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:39395
29372024-12-13T22:41:16.067ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38191
29382024-12-13T22:41:16.067ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 64 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38191 req_id = 00ab811a-85a6-40bf-aa2b-a9aeacf36a18 response_code = 200 uri = /vpc-routes
29392024-12-13T22:41:16.135ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {b9fc0549-4550-46e9-aa06-4737b874563c: {10.151.2.176: {e2f6a8ee-b12d-4bea-ac9e-60aff07a0518}, 10.151.2.175: {e2f6a8ee-b12d-4bea-ac9e-60aff07a0518}}} }
29402024-12-13T22:41:16.135ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 127 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:38191 req_id = 8e834f64-7990-4ec7-9e5c-273a6ef4b39a response_code = 204 uri = /eip-gateways
29412024-12-13T22:41:16.217ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:46828
29422024-12-13T22:41:16.237ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:58691
29432024-12-13T22:41:16.239ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 56 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:58691 req_id = 109fbed3-06a4-4491-bf06-6b6554e955ce response_code = 200 uri = /vpc-routes
29442024-12-13T22:41:16.258ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:46474
29452024-12-13T22:41:16.258ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 141 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:46474 req_id = 2fd6e2da-71ea-4a84-b1f8-0908b00f24c2 response_code = 200 uri = /v2p
29462024-12-13T22:41:16.271ZWARNSledAgent (InstanceManager): wait for service Some("oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf") failed: Property not found. retry in 1.169541694s file = illumos-utils/src/svc.rs:36 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf zone = oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf
29472024-12-13T22:41:16.272ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 494 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:38191 req_id = 8c239cf9-8733-48d7-8845-ab1763241d5f response_code = 204 uri = /vpc-routes
29482024-12-13T22:41:16.278ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:54477
29492024-12-13T22:41:16.278ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
29502024-12-13T22:41:16.320ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
29512024-12-13T22:41:16.388ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 109692 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::c]:54477 req_id = c8a72ec7-b87c-47b0-a7d7-314f09e1bed0 response_code = 204 uri = /switch-ports
29522024-12-13T22:41:16.388ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {b9fc0549-4550-46e9-aa06-4737b874563c: {10.151.2.175: {e2f6a8ee-b12d-4bea-ac9e-60aff07a0518}, 10.151.2.176: {e2f6a8ee-b12d-4bea-ac9e-60aff07a0518}}} }
29532024-12-13T22:41:16.388ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:32910
29542024-12-13T22:41:16.388ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 167 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:58691 req_id = 3385d4da-90a0-46b5-8a25-c4f0c36fc380 response_code = 204 uri = /eip-gateways
29552024-12-13T22:41:16.388ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::b]:60159
29562024-12-13T22:41:16.388ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 244 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:54477 req_id = f21cb03a-fb18-420b-9499-503880f3a4fa response_code = 200 uri = /network-bootstore-config
29572024-12-13T22:41:16.388ZINFOSledAgent (ServiceManager): ensuring scrimlet uplinks file = sled-agent/src/services.rs:4119
29582024-12-13T22:41:16.392ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 4135 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:60159 req_id = cac75124-fa27-42e4-91bd-e9597d6fe226 response_code = 200 uri = /v2p
29592024-12-13T22:41:16.429ZINFOSledAgent (ServiceManager): setting up port_qsfp0 file = sled-agent/src/services.rs:4143
29602024-12-13T22:41:16.469ZINFOSledAgent (PortManager): skipping delta compute for subnet file = illumos-utils/src/opte/port_manager.rs:486 new_vers = RouterVersion { router_id: 144e76a3-7626-4a11-b3fd-c0ec234f7023, version: 3 } old_vers = RouterVersion { router_id: 144e76a3-7626-4a11-b3fd-c0ec234f7023, version: 3 } subnet = RouterId { vni: Vni(14180905), kind: System }
29612024-12-13T22:41:16.469ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 567 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::b]:58691 req_id = efc3ed94-6748-4fe1-bec2-12472f68a69e response_code = 204 uri = /vpc-routes
29622024-12-13T22:41:16.497ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 108999 local_addr = [fd00:1122:3344:101::1]:12345 method = POST remote_addr = [fd00:1122:3344:101::b]:32910 req_id = 4992ed72-6797-43a9-b474-93011a716427 response_code = 204 uri = /switch-ports
29632024-12-13T22:41:16.498ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 144 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:32910 req_id = 9fa204af-28f7-4144-a347-c5fe3f7748af response_code = 200 uri = /network-bootstore-config
29642024-12-13T22:41:17.472ZINFOSledAgent (InstanceManager): Started propolis in zone: oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf file = sled-agent/src/instance.rs:1786 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
29652024-12-13T22:41:17.482ZINFOSledAgent (InstanceManager): Propolis SMF service is online file = sled-agent/src/instance.rs:1795 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
29662024-12-13T22:41:17.519ZINFOSledAgent (InstanceManager): Propolis HTTP server online file = sled-agent/src/instance.rs:1819 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
29672024-12-13T22:41:17.660ZINFOSledAgent (InstanceManager): result of instance_ensure call is Ok(InstanceEnsureResponse { migrate: None }) file = sled-agent/src/instance.rs:932 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
29682024-12-13T22:41:17.661ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1240 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
29692024-12-13T22:41:17.661ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1444328 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::b]:46828 req_id = 54b8d449-cd9d-4b55-84c2-d6b705625f77 response_code = 200 uri = /vmms/e491dee3-b7b4-4af4-a879-4c884a189dbf/state
29702024-12-13T22:41:17.661ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 2715275 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:62006 req_id = 76c1d484-0b45-4b92-9d27-9daaafa12033 response_code = 200 uri = /vmms/e491dee3-b7b4-4af4-a879-4c884a189dbf/state
29712024-12-13T22:41:17.661ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 7557176 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55205 req_id = 2f84288b-1c17-4349-9064-bdc59511abb6 response_code = 204 uri = /eip-gateways
29722024-12-13T22:41:17.661ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 1639980 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:39395 req_id = fc7c6b8d-c809-4b09-a7cb-d7a6079c320c response_code = 200 uri = /vmms/e491dee3-b7b4-4af4-a879-4c884a189dbf/state
29732024-12-13T22:41:17.662ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Creating), migration_in: None, migration_out: None, time: 2024-12-13T22:41:17.662943318Z } file = sled-agent/src/instance.rs:780 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
29742024-12-13T22:41:17.662ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 new_vmm_state = VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-12-13T22:41:17.662943318Z } propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
29752024-12-13T22:41:17.662ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(3), time_updated: 2024-12-13T22:41:17.662943318Z }, migration_in: None, migration_out: None }
29762024-12-13T22:41:17.712ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Running), migration_in: None, migration_out: None, time: 2024-12-13T22:41:17.712618051Z } file = sled-agent/src/instance.rs:780 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
29772024-12-13T22:41:17.712ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 new_vmm_state = VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-12-13T22:41:17.712618051Z } propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
29782024-12-13T22:41:17.712ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf state = SledVmmState { vmm_state: VmmRuntimeState { state: Running, gen: Generation(4), time_updated: 2024-12-13T22:41:17.712618051Z }, migration_in: None, migration_out: None }
29792024-12-13T22:41:17.750ZINFOSledAgent (PortManager): skipping delta compute for subnet file = illumos-utils/src/opte/port_manager.rs:486 new_vers = RouterVersion { router_id: 144e76a3-7626-4a11-b3fd-c0ec234f7023, version: 3 } old_vers = RouterVersion { router_id: 144e76a3-7626-4a11-b3fd-c0ec234f7023, version: 3 } subnet = RouterId { vni: Vni(14180905), kind: System }
29802024-12-13T22:41:17.750ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 206 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:55205 req_id = 74fffc8c-0438-4170-aae9-4199138dbd46 response_code = 204 uri = /vpc-routes
29812024-12-13T22:41:17.792ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48793
29822024-12-13T22:41:17.792ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 52 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:48793 req_id = 6105d764-06e2-4a80-9db6-08a6c1cf3306 response_code = 200 uri = /vpc-routes
29832024-12-13T22:41:17.839ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {b9fc0549-4550-46e9-aa06-4737b874563c: {10.151.2.175: {e2f6a8ee-b12d-4bea-ac9e-60aff07a0518}, 10.151.2.176: {e2f6a8ee-b12d-4bea-ac9e-60aff07a0518}}} }
29842024-12-13T22:41:17.839ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 102 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48793 req_id = e7784865-0073-40ce-87ef-e9870e80be2f response_code = 204 uri = /eip-gateways
29852024-12-13T22:41:17.853ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 248 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48793 req_id = 4edc97e3-48b8-41b7-bf95-2cf424a26a79 response_code = 204 uri = /vpc-routes
29862024-12-13T22:41:28.405ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:48080
29872024-12-13T22:41:28.406ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 445 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:48080 req_id = d24cdffa-dd21-451e-8a67-1858735c455d response_code = 200 uri = /vmms/e491dee3-b7b4-4af4-a879-4c884a189dbf/state
29882024-12-13T22:41:28.406ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Stopping), migration_in: None, migration_out: None, time: 2024-12-13T22:41:28.406559877Z } file = sled-agent/src/instance.rs:780 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
29892024-12-13T22:41:28.406ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 new_vmm_state = VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-12-13T22:41:28.406559877Z } propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
29902024-12-13T22:41:28.406ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf state = SledVmmState { vmm_state: VmmRuntimeState { state: Stopping, gen: Generation(6), time_updated: 2024-12-13T22:41:28.406559877Z }, migration_in: None, migration_out: None }
29912024-12-13T22:41:28.441ZINFOSledAgent (InstanceManager): Observing new propolis state: ObservedPropolisState { vmm_state: PropolisInstanceState(Destroyed), migration_in: None, migration_out: None, time: 2024-12-13T22:41:28.441526597Z } file = sled-agent/src/instance.rs:780 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
29922024-12-13T22:41:28.441ZINFOSledAgent (InstanceManager): updated state after observing Propolis state change file = sled-agent/src/instance.rs:802 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 new_vmm_state = VmmRuntimeState { state: Destroyed, gen: Generation(7), time_updated: 2024-12-13T22:41:28.441526597Z } propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
29932024-12-13T22:41:28.441ZINFOSledAgent (InstanceManager): terminating VMM that has exited file = sled-agent/src/instance.rs:816 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
29942024-12-13T22:41:28.441ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
29952024-12-13T22:41:28.441ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:602 id = 4813595674667834954 n_samples = 6 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
29962024-12-13T22:41:28.441ZWARNSledAgent (kstat-sampler-worker): target removed with queued samples file = oximeter/instruments/src/kstat/sampler.rs:602 id = 2986646974009255898 n_samples = 6 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
29972024-12-13T22:41:28.441ZINFOSledAgent (StorageManager): Received GetLatestResources(...) file = sled-storage/src/manager.rs:603
29982024-12-13T22:41:28.441ZINFOSledAgent (ZoneBundler): creating zone bundle context = ZoneBundleContext { storage_dirs: ["/pool/int/6c407cb8-0013-4482-9b28-b5bd52e5b10f/debug/bundle/zone", "/pool/int/dbb4b417-a6b8-4743-be7a-9642ba91ec5c/debug/bundle/zone"], cause: TerminatedInstance, extra_log_dirs: ["/pool/ext/d3dcd055-d94b-4283-a632-f7eab7bbead5/crypt/debug", "/pool/ext/0d689a16-0a28-4b27-af5a-3b4bf99f365f/crypt/debug", "/pool/ext/9e30dd3c-3e72-4b01-8f5f-662db99dc37a/crypt/debug", "/pool/ext/8159e296-b0fc-4958-9b6f-9386d6e003fa/crypt/debug", "/pool/ext/fb594314-d689-4df6-8176-e09675ab8f03/crypt/debug", "/pool/ext/5bc1d750-0ca6-4a08-bf34-a01bf34cccef/crypt/debug", "/pool/ext/b80657b8-dd44-496f-8b60-edeffe6ea0c4/crypt/debug", "/pool/ext/35c96663-c574-4d07-a692-3eb8a2a79302/crypt/debug", "/pool/ext/ffd405c4-ad68-4401-9d01-3a9d70b7775f/crypt/debug"] } file = sled-agent/src/zone_bundle.rs:339 zone_name = oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf
29992024-12-13T22:41:29.088ZINFOSledAgent (ZoneBundler): finished zone bundle file = sled-agent/src/zone_bundle.rs:1142 metadata = ZoneBundleMetadata { id: ZoneBundleId { zone_name: "oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf", bundle_id: 96a79a95-98cf-440f-beba-50bacf3f6963 }, time_created: 2024-12-13T22:41:28.443437481Z, version: 0, cause: TerminatedInstance }
30002024-12-13T22:41:29.088ZWARNSledAgent (InstanceManager): Halting and removing zone: oxz_propolis-server_e491dee3-b7b4-4af4-a879-4c884a189dbf file = sled-agent/src/instance.rs:1032 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
30012024-12-13T22:41:30.843ZINFOSledAgent (InstanceManager): halt_and_remove_logged: Previous zone state: Running file = illumos-utils/src/zone.rs:279 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
30022024-12-13T22:41:30.861ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:1000 id = RouterId { vni: Vni(14180905), kind: System }
30032024-12-13T22:41:30.861ZINFOSledAgent (PortManager): Removed route set for subnet file = illumos-utils/src/opte/port_manager.rs:1000 id = RouterId { vni: Vni(14180905), kind: Custom(V4(Ipv4Net { addr: 172.30.0.0, width: 22 })) }
30042024-12-13T22:41:30.862ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-12-13T22:41:30.862035285Z }, migration_in: None, migration_out: None }
30052024-12-13T22:41:30.890ZINFOSledAgent (InstanceManager): Publishing instance state update to Nexus file = sled-agent/src/instance.rs:700 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf state = SledVmmState { vmm_state: VmmRuntimeState { state: Destroyed, gen: Generation(8), time_updated: 2024-12-13T22:41:30.862035285Z }, migration_in: None, migration_out: None }
30062024-12-13T22:41:30.894ZINFOSledAgent (InstanceManager): State monitoring task complete file = sled-agent/src/instance.rs:969 instance_id = ae4fa161-b162-4506-a738-be9e4cbbf9c6 propolis_id = e491dee3-b7b4-4af4-a879-4c884a189dbf
30072024-12-13T22:41:31.400ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:51215
30082024-12-13T22:41:31.400ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 106 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:51215 req_id = a1c551ad-0374-4848-8647-f1eb2ff89a01 response_code = 200 uri = /vpc-routes
30092024-12-13T22:41:31.440ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::c]:38325
30102024-12-13T22:41:31.440ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 143 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::c]:38325 req_id = f7b943eb-4c7d-48cb-91dc-5453bc1ab22e response_code = 200 uri = /v2p
30112024-12-13T22:41:31.443ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
30122024-12-13T22:41:31.443ZINFOSledAgent: IGW mapping changed; external IPs refreshed file = sled-agent/src/sled_agent.rs:1240 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765
30132024-12-13T22:41:31.443ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 126 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51215 req_id = 3ffd615a-16f6-45b1-995f-384eccc0ae5d response_code = 204 uri = /eip-gateways
30142024-12-13T22:41:31.451ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 381 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::c]:51215 req_id = 609ead0e-4033-44af-a12d-0a2a14a11e41 response_code = 204 uri = /vpc-routes
30152024-12-13T22:41:31.608ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:64074
30162024-12-13T22:41:31.608ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 156 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:64074 req_id = b161b5ee-a977-44ae-9bff-d8ec898853e1 response_code = 200 uri = /v2p
30172024-12-13T22:41:31.726ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:50981
30182024-12-13T22:41:31.726ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 77 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:50981 req_id = a186caf8-4917-4fce-8f87-445d6a9bc8a8 response_code = 200 uri = /vpc-routes
30192024-12-13T22:41:31.740ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
30202024-12-13T22:41:31.740ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 51 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50981 req_id = e66e7694-8d7b-4361-9649-87ee048c9898 response_code = 204 uri = /eip-gateways
30212024-12-13T22:41:31.748ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 270 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:50981 req_id = e39a7de6-3227-4c69-bcf3-878b2512b5a8 response_code = 204 uri = /vpc-routes
30222024-12-13T22:41:32.306ZINFOSledAgent (dropshot (SledAgent)): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:1023 local_addr = [fd00:1122:3344:101::1]:12345 remote_addr = [fd00:1122:3344:101::a]:37279
30232024-12-13T22:41:32.307ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 88 local_addr = [fd00:1122:3344:101::1]:12345 method = GET remote_addr = [fd00:1122:3344:101::a]:37279 req_id = 1bd3106d-cf58-43b9-9d18-e942e18a5858 response_code = 200 uri = /vpc-routes
30242024-12-13T22:41:32.320ZINFOSledAgent: IGW mapping received file = sled-agent/src/sled_agent.rs:1214 sled_id = 35eb16d3-adec-4806-a15c-ab7a15d18765 values = ExternalIpGatewayMap { mappings: {} }
30252024-12-13T22:41:32.320ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 79 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:37279 req_id = df6fc11a-4e00-4f54-9fc8-e75c336dd608 response_code = 204 uri = /eip-gateways
30262024-12-13T22:41:32.328ZINFOSledAgent (dropshot (SledAgent)): request completed file = /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/dropshot-0.13.0/src/server.rs:863 latency_us = 249 local_addr = [fd00:1122:3344:101::1]:12345 method = PUT remote_addr = [fd00:1122:3344:101::a]:37279 req_id = af3805d4-5e9a-45aa-879e-ff1452dc2425 response_code = 204 uri = /vpc-routes